2024-09-15T01:31:28.238 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T01:31:28.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:31:28.339 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} email: skanta@redhat.com first_in_suite: false job_id: '7905049' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 2500 ms tcp read timeout: 5 osd_async_recovery_min_cost: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 9 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - mon.a - - osd.4 - osd.5 - osd.6 - osd.7 - mon.b - mgr.x - - osd.8 - osd.9 - osd.10 - osd.11 - mon.c seed: 2126 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 3 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 4 l: 3 m: 2 name: lrcprofile plugin: lrc objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_00:54:39 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T01:31:28.340 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-15T01:31:28.340 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-15T01:31:28.340 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T01:31:28.342 INFO:teuthology.task.internal:Checking packages... 2024-09-15T01:31:28.364 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-15T01:31:28.365 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T01:31:28.365 INFO:teuthology.packaging:ref: None 2024-09-15T01:31:28.365 INFO:teuthology.packaging:tag: None 2024-09-15T01:31:28.365 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:31:28.365 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:28.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:31:28.511 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:31:28.513 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T01:31:28.519 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T01:31:28.519 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T01:31:28.550 INFO:teuthology.task.internal:Saving configuration 2024-09-15T01:31:28.566 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T01:31:28.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T01:31:28.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:25:08.521168', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T01:31:28.613 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:25:08.525186', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T01:31:28.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:25:08.524042', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T01:31:28.630 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T01:31:28.636 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2024-09-15T01:31:28.636 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2024-09-15T01:31:28.636 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2024-09-15T01:31:28.636 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T01:31:28.737 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6346cfbac0>, signals=[15]) 2024-09-15T01:31:28.738 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T01:31:28.744 INFO:teuthology.task.internal:Opening connections... 2024-09-15T01:31:28.744 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:31:28.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:31:28.843 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:31:28.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:31:28.929 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:31:28.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:31:29.018 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T01:31:29.025 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-09-15T01:31:29.032 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-09-15T01:31:29.032 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-09-15T01:31:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Ubuntu" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:ID=ubuntu 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE=debian 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="20.04" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_CODENAME=focal 2024-09-15T01:31:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:UBUNTU_CODENAME=focal 2024-09-15T01:31:29.083 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-09-15T01:31:29.105 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-15T01:31:29.112 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-15T01:31:29.112 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-15T01:31:29.159 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Ubuntu" 2024-09-15T01:31:29.159 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T01:31:29.159 INFO:teuthology.orchestra.run.smithi047.stdout:ID=ubuntu 2024-09-15T01:31:29.159 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE=debian 2024-09-15T01:31:29.159 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="20.04" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_CODENAME=focal 2024-09-15T01:31:29.160 INFO:teuthology.orchestra.run.smithi047.stdout:UBUNTU_CODENAME=focal 2024-09-15T01:31:29.160 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-15T01:31:29.181 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-09-15T01:31:29.187 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-09-15T01:31:29.187 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-09-15T01:31:29.236 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="20.04" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=focal 2024-09-15T01:31:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=focal 2024-09-15T01:31:29.238 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-09-15T01:31:29.255 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T01:31:29.265 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T01:31:29.352 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T01:31:29.352 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:31:29.354 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:31:29.357 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:31:29.363 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T01:31:29.458 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T01:31:29.458 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:31:29.460 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:31:29.462 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:31:29.480 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T01:31:29.586 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T01:31:29.601 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T01:31:29.601 INFO:teuthology.task.kernel:config {'osd.0': {'kdb': 1, 'sha1': 'distro'}, 'osd.4': {'kdb': 1, 'sha1': 'distro'}, 'osd.8': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T01:31:29.601 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:31:29.601 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:31:29.602 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:31:29.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:31:29.607 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-15T01:31:29.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:31:29.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:31:29.609 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-09-15T01:31:29.610 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-15T01:31:29.616 INFO:teuthology.orchestra.run.smithi047.stdout:5.4.0-124-generic 2024-09-15T01:31:29.616 INFO:teuthology.orchestra.run.smithi155.stdout:5.4.0-124-generic 2024-09-15T01:31:29.616 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.4.0-124-generic 2024-09-15T01:31:29.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2024-09-15T01:31:29.617 INFO:teuthology.task.kernel:Running kernel on smithi155: 5.4.0-124-generic 2024-09-15T01:31:29.617 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-09-15T01:31:29.618 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-124-generic 2024-09-15T01:31:29.618 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.4.0-124-generic 2024-09-15T01:31:29.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-09-15T01:31:29.722 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-09-15T01:31:29.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-15T01:31:29.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-09-15T01:31:29.883 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:31:29.888 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:31:29.890 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T01:31:29.898 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T01:31:29.954 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T01:31:29.958 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T01:31:29.991 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T01:31:29.997 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T01:31:30.088 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:31:30.089 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T01:31:30.114 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T01:31:30.272 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T01:31:30.299 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T01:31:30.377 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T01:31:30.418 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T01:31:30.422 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T01:31:30.422 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T01:31:30.426 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T01:31:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T01:31:30.444 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T01:31:30.445 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T01:31:30.446 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T01:31:30.447 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T01:31:30.447 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T01:31:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T01:31:30.460 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T01:31:30.467 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T01:31:30.469 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T01:31:30.469 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T01:31:30.470 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T01:31:30.470 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T01:31:30.470 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T01:31:30.471 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T01:31:30.471 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T01:31:30.471 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T01:31:30.473 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:30.473 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T01:31:30.477 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T01:31:30.481 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T01:31:30.487 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T01:31:30.487 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T01:31:30.487 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T01:31:30.488 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T01:31:30.489 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T01:31:30.489 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T01:31:30.489 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T01:31:30.490 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T01:31:30.491 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T01:31:30.491 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T01:31:30.492 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T01:31:30.503 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T01:31:30.508 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T01:31:30.508 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T01:31:30.509 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T01:31:30.509 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T01:31:30.558 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T01:31:30.615 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T01:31:30.621 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T01:31:30.708 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T01:31:30.907 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T01:31:31.098 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T01:31:31.101 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T01:31:31.106 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T01:31:31.113 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T01:31:31.113 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T01:31:31.115 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T01:31:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T01:31:31.123 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T01:31:31.123 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T01:31:31.124 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T01:31:31.129 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T01:31:31.129 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T01:31:31.175 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T01:31:31.192 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T01:31:31.200 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T01:31:31.201 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T01:31:31.203 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T01:31:31.204 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T01:31:31.213 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T01:31:31.217 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T01:31:31.218 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:31.218 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T01:31:31.218 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T01:31:31.220 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:31.220 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T01:31:31.226 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:31.226 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T01:31:31.238 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T01:31:31.263 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T01:31:31.271 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T01:31:31.271 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T01:31:31.271 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T01:31:31.272 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T01:31:31.277 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T01:31:31.280 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T01:31:31.284 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T01:31:31.285 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T01:31:31.285 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T01:31:31.285 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T01:31:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T01:31:31.290 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T01:31:31.291 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T01:31:31.292 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T01:31:31.298 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T01:31:31.298 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T01:31:31.342 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T01:31:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T01:31:31.347 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T01:31:31.360 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T01:31:31.369 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T01:31:31.374 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T01:31:31.375 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T01:31:31.375 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T01:31:31.439 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T01:31:31.439 INFO:teuthology.orchestra.run.smithi047.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T01:31:31.440 INFO:teuthology.orchestra.run.smithi047.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T01:31:31.440 INFO:teuthology.orchestra.run.smithi047.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T01:31:31.442 INFO:teuthology.orchestra.run.smithi047.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T01:31:31.520 INFO:teuthology.orchestra.run.smithi047.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T01:31:31.521 INFO:teuthology.orchestra.run.smithi047.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T01:31:31.521 INFO:teuthology.orchestra.run.smithi047.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T01:31:31.756 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T01:31:31.756 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T01:31:31.757 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T01:31:31.757 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T01:31:34.840 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 21.8 MB in 3s (7,095 kB/s) 2024-09-15T01:31:34.985 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 21.8 MB in 3s (6,939 kB/s) 2024-09-15T01:31:35.645 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 21.8 MB in 4s (5,811 kB/s) 2024-09-15T01:31:36.324 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:31:36.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:31:36.420 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:31:36.574 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:31:36.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:31:36.637 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-09-15T01:31:36.638 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-09-15T01:31:36.680 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:31:36.843 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-09-15T01:31:36.843 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:36.843 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T01:31:36.844 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:36.845 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-09-15T01:31:36.845 INFO:teuthology.orchestra.run.smithi155.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T01:31:36.845 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-09-15T01:31:36.845 INFO:teuthology.orchestra.run.smithi155.stdout: thermald 2024-09-15T01:31:36.909 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-09-15T01:31:36.909 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:36.909 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T01:31:36.909 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:36.910 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-09-15T01:31:36.911 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T01:31:36.912 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-15T01:31:36.912 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-15T01:31:37.094 INFO:teuthology.orchestra.run.smithi155.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:31:37.095 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 77.2 MB of archives. 2024-09-15T01:31:37.095 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T01:31:37.095 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T01:31:37.121 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-09-15T01:31:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T01:31:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:37.123 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-09-15T01:31:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T01:31:37.123 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-09-15T01:31:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: thermald 2024-09-15T01:31:37.190 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-09-15T01:31:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T01:31:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:37.191 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-09-15T01:31:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T01:31:37.232 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:31:37.245 INFO:teuthology.orchestra.run.smithi033.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:31:37.245 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 77.2 MB of archives. 2024-09-15T01:31:37.245 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T01:31:37.245 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T01:31:37.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:31:37.329 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:31:37.553 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-09-15T01:31:37.553 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-09-15T01:31:37.584 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T01:31:37.677 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T01:31:37.765 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-09-15T01:31:37.765 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:37.765 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T01:31:37.766 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:37.767 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-09-15T01:31:37.767 INFO:teuthology.orchestra.run.smithi047.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T01:31:37.767 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2024-09-15T01:31:37.767 INFO:teuthology.orchestra.run.smithi047.stdout: thermald 2024-09-15T01:31:37.824 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-09-15T01:31:37.824 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T01:31:37.825 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T01:31:37.825 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T01:31:37.826 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2024-09-15T01:31:37.827 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T01:31:37.879 INFO:teuthology.orchestra.run.smithi047.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:31:37.880 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 77.2 MB of archives. 2024-09-15T01:31:37.880 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T01:31:37.880 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T01:31:38.166 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T01:31:38.166 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T01:31:38.166 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T01:31:38.172 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T01:31:38.226 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T01:31:38.262 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T01:31:38.263 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T01:31:38.273 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T01:31:38.496 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T01:31:38.716 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T01:31:38.716 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T01:31:38.716 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T01:31:38.811 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T01:31:38.822 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T01:31:38.980 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 77.2 MB in 1s (71.9 MB/s) 2024-09-15T01:31:39.176 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T01:31:39.542 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 77.2 MB in 1s (78.1 MB/s) 2024-09-15T01:31:39.715 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T01:31:39.962 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T01:31:39.962 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T01:31:39.963 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T01:31:40.255 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T01:31:40.294 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T01:31:40.920 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T01:31:40.925 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:41.002 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 77.2 MB in 3s (22.9 MB/s) 2024-09-15T01:31:41.106 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:41.190 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T01:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T01:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:41.587 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:42.855 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T01:31:42.860 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:42.945 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:43.099 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T01:31:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:43.207 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T01:31:43.519 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:43.619 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:44.336 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T01:31:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:44.394 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:44.723 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T01:31:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:44.781 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:44.812 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T01:31:44.825 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:44.922 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:45.965 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T01:31:45.979 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:46.032 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:49.250 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:49.419 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:49.561 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:49.687 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:49.779 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:49.905 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:50.047 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:50.174 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:50.259 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T01:31:50.273 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T01:31:50.317 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:31:50.553 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T01:31:50.567 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T01:31:50.611 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:31:50.644 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:50.769 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:51.122 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:51.247 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:51.609 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T01:31:51.624 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T01:31:51.667 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:31:55.850 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T01:31:55.873 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:55.908 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:56.534 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T01:31:56.556 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:56.609 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:57.855 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T01:31:57.875 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T01:31:57.912 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:58.709 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:58.812 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:59.143 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:31:59.363 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:31:59.493 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:31:59.868 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:00.636 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T01:32:00.764 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T01:32:01.000 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:01.089 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:01.131 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:02.192 INFO:teuthology.orchestra.run.smithi033.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:02.192 INFO:teuthology.orchestra.run.smithi033.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:02.277 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:02.311 INFO:teuthology.orchestra.run.smithi047.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:02.311 INFO:teuthology.orchestra.run.smithi047.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:02.335 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:02.450 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:03.422 INFO:teuthology.orchestra.run.smithi155.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:03.422 INFO:teuthology.orchestra.run.smithi155.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:03.554 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:32:03.562 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:03.659 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:32:03.713 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T01:32:03.810 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T01:32:03.839 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:03.936 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:03.965 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T01:32:04.079 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T01:32:04.116 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T01:32:04.205 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T01:32:04.234 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:04.303 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T01:32:04.303 INFO:teuthology.orchestra.run.smithi033.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:04.357 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:04.434 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T01:32:04.434 INFO:teuthology.orchestra.run.smithi047.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:04.747 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T01:32:04.873 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T01:32:05.007 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:05.125 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T01:32:05.234 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T01:32:05.335 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T01:32:05.415 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T01:32:05.416 INFO:teuthology.orchestra.run.smithi155.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:27.280 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T01:32:27.280 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub' 2024-09-15T01:32:27.294 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:Generating grub configuration file ... 2024-09-15T01:32:27.618 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:27.640 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:27.649 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T01:32:27.649 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub' 2024-09-15T01:32:27.663 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:27.686 INFO:teuthology.orchestra.run.smithi047.stdout:Generating grub configuration file ... 2024-09-15T01:32:27.745 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T01:32:27.745 INFO:teuthology.orchestra.run.smithi155.stdout:Sourcing file `/etc/default/grub' 2024-09-15T01:32:27.768 INFO:teuthology.orchestra.run.smithi155.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:27.788 INFO:teuthology.orchestra.run.smithi155.stdout:Generating grub configuration file ... 2024-09-15T01:32:28.006 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:28.009 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:28.021 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:28.027 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:28.104 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:28.420 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:28.431 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:28.509 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:28.519 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:30.175 INFO:teuthology.orchestra.run.smithi033.stdout:done 2024-09-15T01:32:30.300 INFO:teuthology.orchestra.run.smithi155.stdout:done 2024-09-15T01:32:30.620 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-09-15T01:32:30.653 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-09-15T01:32:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:30.664 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-09-15T01:32:30.665 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T01:32:30.665 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:32:30.665 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:32:30.666 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2024-09-15T01:32:30.666 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:32:30.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-09-15T01:32:30.712 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-09-15T01:32:30.766 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-09-15T01:32:30.766 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-09-15T01:32:30.766 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-09-15T01:32:30.766 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-09-15T01:32:30.766 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 20 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:30.767 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-09-15T01:32:30.767 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T01:32:30.768 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:32:30.768 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:32:30.768 INFO:teuthology.task.kernel:Installing distro kernel on osd.8... 2024-09-15T01:32:30.768 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:32:30.768 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-09-15T01:32:30.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-15T01:32:30.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-09-15T01:32:30.920 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:32:30.926 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:32:30.944 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:32:30.997 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:32:31.005 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:32:31.023 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:32:31.038 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:32:31.090 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2024-09-15T01:32:31.116 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:32:31.128 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2024-09-15T01:32:31.128 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 20 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2024-09-15T01:32:31.130 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T01:32:31.130 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:32:31.130 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:32:31.130 INFO:teuthology.task.kernel:Installing distro kernel on osd.4... 2024-09-15T01:32:31.130 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:32:31.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2024-09-15T01:32:31.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-09-15T01:32:31.381 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:32:31.389 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:32:31.404 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:32:31.500 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:32:32.837 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:32:32.844 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:32:32.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:32:32.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:32:32.933 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:32:32.942 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:32:33.141 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-09-15T01:32:33.142 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-09-15T01:32:33.166 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-15T01:32:33.167 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-15T01:32:33.312 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:32:33.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T01:32:33.375 INFO:teuthology.orchestra.run.smithi155.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T01:32:33.375 INFO:teuthology.orchestra.run.smithi155.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:32:33.377 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 20 2024-09-15T01:32:33.398 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:33.399 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-09-15T01:32:33.399 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-09-15T01:32:33.404 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.wSgm1nBlyU 2024-09-15T01:32:33.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub/grub.cfg /tmp/tmp.wSgm1nBlyU 2024-09-15T01:32:33.418 INFO:teuthology.orchestra.run.smithi033.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T01:32:33.418 INFO:teuthology.orchestra.run.smithi033.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:32:33.419 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.wSgm1nBlyU 2024-09-15T01:32:33.420 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-09-15T01:32:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-09-15T01:32:33.441 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:32:33.494 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.k7ieC29kcz 2024-09-15T01:32:33.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub/grub.cfg /tmp/tmp.k7ieC29kcz 2024-09-15T01:32:33.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.k7ieC29kcz 2024-09-15T01:32:33.510 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.wSgm1nBlyU is 9KB 2024-09-15T01:32:33.526 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.wSgm1nBlyU 2024-09-15T01:32:33.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.588 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:32:33.588 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.588 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.589 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.k7ieC29kcz is 9KB 2024-09-15T01:32:33.603 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.k7ieC29kcz 2024-09-15T01:32:33.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.626 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T01:32:33.627 DEBUG:teuthology.orchestra.run.smithi155:> sudo update-grub 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-09-15T01:32:33.659 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-09-15T01:32:33.667 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:32:33.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:33.703 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T01:32:33.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo update-grub 2024-09-15T01:32:33.899 INFO:teuthology.orchestra.run.smithi047.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T01:32:33.900 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T01:32:33.901 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2024-09-15T01:32:33.923 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2024-09-15T01:32:33.923 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2024-09-15T01:32:33.923 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 20 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.4.0.195.193 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2024-09-15T01:32:33.924 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2024-09-15T01:32:33.925 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T01:32:33.931 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.ofGmJfpJpO 2024-09-15T01:32:33.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ofGmJfpJpO 2024-09-15T01:32:33.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.ofGmJfpJpO 2024-09-15T01:32:34.028 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.ofGmJfpJpO is 9KB 2024-09-15T01:32:34.044 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.ofGmJfpJpO 2024-09-15T01:32:34.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:34.108 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:32:34.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T01:32:34.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T01:32:34.142 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T01:32:34.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo update-grub 2024-09-15T01:32:34.993 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub' 2024-09-15T01:32:35.001 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:35.018 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-09-15T01:32:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub' 2024-09-15T01:32:35.176 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-09-15T01:32:35.279 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:35.291 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub' 2024-09-15T01:32:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T01:32:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-09-15T01:32:35.674 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:35.685 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T01:32:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T01:32:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T01:32:36.427 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-09-15T01:32:36.428 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-09-15T01:32:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-09-15T01:32:36.826 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-09-15T01:32:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-09-15T01:32:37.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-09-15T01:33:06.443 INFO:teuthology.task.kernel:Checking client osd.8 for new kernel version... 2024-09-15T01:33:06.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:33:06.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:06.829 INFO:teuthology.task.kernel:Checking client osd.0 for new kernel version... 2024-09-15T01:33:06.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:33:06.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:07.191 INFO:teuthology.task.kernel:Checking client osd.4 for new kernel version... 2024-09-15T01:33:07.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:33:07.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:24.928 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-09-15T01:33:25.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-15T01:33:25.697 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-09-15T01:33:33.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:33:33.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:34.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:33:34.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:34.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:33:34.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:36.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-09-15T01:33:48.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:33:48.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:33:52.573 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-15T01:34:04.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:34:04.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:34:07.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-15T01:34:22.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:34:22.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:34:22.911 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-15T01:34:23.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:34:23.467 INFO:teuthology.task.kernel:Checking kernel version of osd.0, want "5.4.0-195-generic"... 2024-09-15T01:34:23.468 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-15T01:34:23.515 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-195-generic 2024-09-15T01:34:23.516 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T01:34:23.516 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:34:23.516 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T01:34:24.517 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2024-09-15T01:34:24.517 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:34:24.604 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-09-15T01:34:24.624 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:34.702 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T01:34:46.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:34:46.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:34:46.956 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-15T01:34:47.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:34:47.511 INFO:teuthology.task.kernel:Checking kernel version of osd.4, want "5.4.0-195-generic"... 2024-09-15T01:34:47.512 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-15T01:34:47.518 INFO:teuthology.orchestra.run.smithi047.stdout:5.4.0-195-generic 2024-09-15T01:34:47.518 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T01:34:47.519 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:34:47.519 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T01:34:48.519 INFO:teuthology.task.kernel:Enabling kdb on osd.4... 2024-09-15T01:34:48.520 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:34:48.613 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-09-15T01:34:48.638 DEBUG:teuthology.parallel:result is None 2024-09-15T01:34:49.001 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T01:35:04.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:35:04.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:35:04.165 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-09-15T01:35:04.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:35:04.719 INFO:teuthology.task.kernel:Checking kernel version of osd.8, want "5.4.0-195-generic"... 2024-09-15T01:35:04.719 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-09-15T01:35:04.725 INFO:teuthology.orchestra.run.smithi155.stdout:5.4.0-195-generic 2024-09-15T01:35:04.726 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T01:35:04.726 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:35:04.726 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T01:35:05.727 INFO:teuthology.task.kernel:Enabling kdb on osd.8... 2024-09-15T01:35:05.727 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:35:05.814 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-09-15T01:35:05.833 DEBUG:teuthology.parallel:result is None 2024-09-15T01:35:05.833 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T01:35:05.841 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T01:35:05.841 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:35:05.843 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:35:05.846 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:35:05.853 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T01:35:05.987 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T01:35:06.075 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T01:35:06.075 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:06.078 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:06.081 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:35:06.107 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T01:35:06.147 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T01:35:06.147 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:35:06.150 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:35:06.152 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:35:06.171 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.174 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.181 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.181 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.184 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.190 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:35:06.192 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T01:35:06.262 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T01:35:06.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:35:06.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:35:06.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:35:06.286 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T01:35:06.316 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T01:35:06.316 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:35:06.318 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:35:06.321 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:35:06.333 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:35:06.373 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:35:06.380 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:35:06.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:35:06.401 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:35:06.408 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:35:06.416 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:35:06.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:35:06.437 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:35:06.444 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:35:06.451 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:35:06.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:35:06.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-09-15T01:35:06.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-15T01:35:06.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-09-15T01:35:06.582 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T01:35:06.590 INFO:teuthology.task.internal:Starting timer... 2024-09-15T01:35:06.590 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T01:35:06.636 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T01:35:06.658 DEBUG:teuthology.task.selinux:Excluding smithi033: OS 'ubuntu' does not support SELinux 2024-09-15T01:35:06.658 DEBUG:teuthology.task.selinux:Excluding smithi047: OS 'ubuntu' does not support SELinux 2024-09-15T01:35:06.658 DEBUG:teuthology.task.selinux:Excluding smithi155: OS 'ubuntu' does not support SELinux 2024-09-15T01:35:06.658 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T01:35:06.658 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T01:35:06.658 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T01:35:06.658 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T01:35:06.690 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T01:35:06.693 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T01:35:06.805 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T01:35:06.815 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T01:35:06.816 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi047.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T01:40:46.843 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-09-15T01:40:46.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:40:46.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:40:46.938 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-15T01:40:47.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-15T01:40:47.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:40:47.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:40:47.167 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-15T01:40:47.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T01:40:47.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:40:47.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:40:47.384 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-09-15T01:40:47.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-09-15T01:40:47.511 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T01:40:47.525 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T01:40:47.526 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:40:47.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:40:47.529 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:40:47.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:40:47.532 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:40:47.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:40:47.557 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T01:40:47.557 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Command line: ntpd -gq 2024-09-15T01:40:47.558 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: proto: precision = 0.087 usec (-23) 2024-09-15T01:40:47.558 INFO:teuthology.orchestra.run.smithi033.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.559 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.559 INFO:teuthology.orchestra.run.smithi033.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.560 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.560 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen normally on 3 ens1f0 172.21.15.33:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen normally on 4 lo [::1]:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:d294%4]:123 2024-09-15T01:40:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:47 ntpd[13719]: Listening on routing socket on fd #22 for interface updates 2024-09-15T01:40:47.565 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T01:40:47.565 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Command line: ntpd -gq 2024-09-15T01:40:47.566 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: proto: precision = 0.227 usec (-22) 2024-09-15T01:40:47.566 INFO:teuthology.orchestra.run.smithi047.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.566 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.566 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.567 INFO:teuthology.orchestra.run.smithi047.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.568 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T01:40:47.568 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T01:40:47.568 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T01:40:47.568 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen normally on 3 ens1f0 172.21.15.47:123 2024-09-15T01:40:47.569 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen normally on 4 lo [::1]:123 2024-09-15T01:40:47.569 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:148e%3]:123 2024-09-15T01:40:47.569 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:47 ntpd[13734]: Listening on routing socket on fd #22 for interface updates 2024-09-15T01:40:47.570 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T01:40:47.570 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Command line: ntpd -gq 2024-09-15T01:40:47.570 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: proto: precision = 0.070 usec (-24) 2024-09-15T01:40:47.570 INFO:teuthology.orchestra.run.smithi155.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.571 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T01:40:47.571 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.571 INFO:teuthology.orchestra.run.smithi155.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T01:40:47.572 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T01:40:47.572 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T01:40:47.572 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T01:40:47.572 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen normally on 3 enp3s0f1 172.21.15.155:123 2024-09-15T01:40:47.572 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen normally on 4 lo [::1]:123 2024-09-15T01:40:47.573 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6ded%5]:123 2024-09-15T01:40:47.573 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:47 ntpd[13758]: Listening on routing socket on fd #22 for interface updates 2024-09-15T01:40:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:48 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:49.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:49 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:50 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:50 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:50.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:50 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:51.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:51 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:51 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:51.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:51.571 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:51.571 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:51.571 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:51.571 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:51 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:52 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:52 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:52.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:52 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:53.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:53 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:53.565 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.565 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:53.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:53 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:53.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:53.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:53 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stdout:15 Sep 01:40:54 ntpd[13719]: ntpd: time slew +0.001364 s 2024-09-15T01:40:54.559 INFO:teuthology.orchestra.run.smithi033.stdout:ntpd: time slew +0.001364s 2024-09-15T01:40:54.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T01:40:54.560 INFO:teuthology.orchestra.run.smithi033.stderr:15 Sep 01:40:54 ntpd[13719]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:54 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:54.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:54 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:54.627 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:40:54.627 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-09-15T01:40:54.628 INFO:teuthology.orchestra.run.smithi033.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:54.628 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:54.628 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:54.628 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:55 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:55.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:55.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:55 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi047.stdout:15 Sep 01:40:56 ntpd[13734]: ntpd: time slew +0.000301 s 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi047.stdout:ntpd: time slew +0.000301s 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:15 Sep 01:40:56 ntpd[13734]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stdout:15 Sep 01:40:56 ntpd[13758]: ntpd: time slew +0.000645 s 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi155.stdout:ntpd: time slew +0.000645s 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi155.stderr:15 Sep 01:40:56 ntpd[13758]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T01:40:56.635 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:40:56.635 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-09-15T01:40:56.636 INFO:teuthology.orchestra.run.smithi155.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.636 INFO:teuthology.orchestra.run.smithi155.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.636 INFO:teuthology.orchestra.run.smithi155.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.636 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.644 INFO:teuthology.orchestra.run.smithi047.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:40:56.645 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================== 2024-09-15T01:40:56.645 INFO:teuthology.orchestra.run.smithi047.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.645 INFO:teuthology.orchestra.run.smithi047.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.645 INFO:teuthology.orchestra.run.smithi047.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.645 INFO:teuthology.orchestra.run.smithi047.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:40:56.646 INFO:teuthology.run_tasks:Running task install... 2024-09-15T01:40:56.655 DEBUG:teuthology.task.install:project ceph 2024-09-15T01:40:56.655 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:40:56.655 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-15T01:40:56.655 INFO:teuthology.task.install:Using flavor: default 2024-09-15T01:40:56.661 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T01:40:56.661 INFO:teuthology.task.install:extra packages: [] 2024-09-15T01:40:56.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-key list | grep Ceph 2024-09-15T01:40:56.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-key list | grep Ceph 2024-09-15T01:40:56.662 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-key list | grep Ceph 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi033.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T01:40:56.788 INFO:teuthology.orchestra.run.smithi047.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T01:40:56.826 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T01:40:56.826 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T01:40:56.827 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T01:40:56.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:40:56.832 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T01:40:56.832 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T01:40:56.833 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T01:40:56.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:40:56.847 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T01:40:56.848 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T01:40:56.848 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T01:40:56.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:40:56.983 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T01:40:56.983 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:57.001 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T01:40:57.001 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:57.005 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T01:40:57.006 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:57.098 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:40:57.098 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T01:40:57.110 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:40:57.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T01:40:57.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-09-15T01:40:57.120 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:40:57.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T01:40:57.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-15T01:40:57.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-09-15T01:40:57.278 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:40:57.282 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:40:57.300 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:40:57.303 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:40:57.303 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:40:57.309 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:40:57.326 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:40:57.330 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T01:40:57.351 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T01:40:57.357 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T01:40:57.368 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T01:40:57.374 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T01:40:57.378 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T01:40:57.390 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T01:40:57.394 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:40:57.395 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T01:40:57.401 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T01:40:57.414 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T01:40:57.423 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T01:40:57.430 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T01:40:57.444 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T01:40:57.444 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T01:40:57.445 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T01:40:57.455 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T01:40:57.478 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T01:40:57.514 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T01:40:57.602 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T01:40:57.941 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 28.5 kB in 1s (44.5 kB/s) 2024-09-15T01:40:57.953 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 28.5 kB in 1s (44.9 kB/s) 2024-09-15T01:40:58.136 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 28.5 kB in 1s (35.9 kB/s) 2024-09-15T01:40:59.114 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:40:59.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:59.146 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:40:59.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:59.214 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-09-15T01:40:59.340 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:40:59.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T01:40:59.431 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-09-15T01:40:59.432 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-09-15T01:40:59.444 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-15T01:40:59.470 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-09-15T01:40:59.471 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-09-15T01:40:59.635 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T01:40:59.636 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T01:40:59.636 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T01:40:59.636 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-09-15T01:40:59.636 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T01:40:59.638 INFO:teuthology.orchestra.run.smithi155.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T01:40:59.639 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: socat xmlstarlet 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T01:40:59.640 INFO:teuthology.orchestra.run.smithi155.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-09-15T01:40:59.641 INFO:teuthology.orchestra.run.smithi155.stdout: btrfs-tools 2024-09-15T01:40:59.669 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-15T01:40:59.670 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-15T01:40:59.680 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T01:40:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T01:40:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T01:40:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T01:40:59.683 INFO:teuthology.orchestra.run.smithi047.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T01:40:59.684 INFO:teuthology.orchestra.run.smithi047.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: socat xmlstarlet 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-09-15T01:40:59.685 INFO:teuthology.orchestra.run.smithi047.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi047.stdout: btrfs-tools 2024-09-15T01:40:59.762 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-09-15T01:40:59.762 INFO:teuthology.orchestra.run.smithi155.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T01:40:59.762 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T01:40:59.762 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T01:40:59.762 INFO:teuthology.orchestra.run.smithi155.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T01:40:59.763 INFO:teuthology.orchestra.run.smithi155.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T01:40:59.763 INFO:teuthology.orchestra.run.smithi155.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T01:40:59.763 INFO:teuthology.orchestra.run.smithi155.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T01:40:59.765 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-09-15T01:40:59.766 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 librbd1 2024-09-15T01:40:59.806 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-09-15T01:40:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T01:40:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T01:40:59.806 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T01:40:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T01:40:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T01:40:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T01:40:59.807 INFO:teuthology.orchestra.run.smithi047.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T01:40:59.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2024-09-15T01:40:59.809 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 librbd1 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi047.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 201 MB of archives. 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi033.stdout: socat xmlstarlet 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi033.stdout: btrfs-tools 2024-09-15T01:40:59.910 INFO:teuthology.orchestra.run.smithi155.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T01:40:59.911 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 201 MB of archives. 2024-09-15T01:40:59.911 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T01:40:59.911 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T01:40:59.958 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T01:40:59.959 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T01:40:59.959 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T01:40:59.960 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T01:40:59.969 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T01:40:59.972 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T01:40:59.999 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T01:41:00.000 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 librbd1 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T01:41:00.058 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T01:41:00.071 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T01:41:00.071 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T01:41:00.072 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T01:41:00.072 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T01:41:00.074 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T01:41:00.074 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T01:41:00.075 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T01:41:00.076 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T01:41:00.077 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T01:41:00.093 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T01:41:00.095 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T01:41:00.096 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T01:41:00.102 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T01:41:00.102 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T01:41:00.102 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T01:41:00.103 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T01:41:00.110 INFO:teuthology.orchestra.run.smithi047.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T01:41:00.111 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T01:41:00.113 INFO:teuthology.orchestra.run.smithi047.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T01:41:00.114 INFO:teuthology.orchestra.run.smithi047.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T01:41:00.128 INFO:teuthology.orchestra.run.smithi047.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T01:41:00.129 INFO:teuthology.orchestra.run.smithi047.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T01:41:00.131 INFO:teuthology.orchestra.run.smithi047.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T01:41:00.133 INFO:teuthology.orchestra.run.smithi047.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T01:41:00.133 INFO:teuthology.orchestra.run.smithi047.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T01:41:00.134 INFO:teuthology.orchestra.run.smithi047.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T01:41:00.179 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T01:41:00.191 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T01:41:00.197 INFO:teuthology.orchestra.run.smithi047.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T01:41:00.206 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T01:41:00.206 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T01:41:00.215 INFO:teuthology.orchestra.run.smithi033.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T01:41:00.215 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 201 MB of archives. 2024-09-15T01:41:00.215 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T01:41:00.215 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T01:41:00.216 INFO:teuthology.orchestra.run.smithi047.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T01:41:00.218 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T01:41:00.219 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T01:41:00.225 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T01:41:00.232 INFO:teuthology.orchestra.run.smithi047.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T01:41:00.233 INFO:teuthology.orchestra.run.smithi047.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T01:41:00.248 INFO:teuthology.orchestra.run.smithi047.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T01:41:00.248 INFO:teuthology.orchestra.run.smithi047.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T01:41:00.248 INFO:teuthology.orchestra.run.smithi047.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T01:41:00.249 INFO:teuthology.orchestra.run.smithi047.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T01:41:00.277 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T01:41:00.296 INFO:teuthology.orchestra.run.smithi047.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T01:41:00.297 INFO:teuthology.orchestra.run.smithi047.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T01:41:00.298 INFO:teuthology.orchestra.run.smithi047.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T01:41:00.299 INFO:teuthology.orchestra.run.smithi047.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T01:41:00.300 INFO:teuthology.orchestra.run.smithi047.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T01:41:00.301 INFO:teuthology.orchestra.run.smithi047.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T01:41:00.305 INFO:teuthology.orchestra.run.smithi047.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T01:41:00.306 INFO:teuthology.orchestra.run.smithi047.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T01:41:00.313 INFO:teuthology.orchestra.run.smithi047.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T01:41:00.325 INFO:teuthology.orchestra.run.smithi047.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T01:41:00.328 INFO:teuthology.orchestra.run.smithi047.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T01:41:00.332 INFO:teuthology.orchestra.run.smithi047.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T01:41:00.350 INFO:teuthology.orchestra.run.smithi047.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T01:41:00.353 INFO:teuthology.orchestra.run.smithi047.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T01:41:00.357 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T01:41:00.362 INFO:teuthology.orchestra.run.smithi047.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T01:41:00.364 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T01:41:00.364 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T01:41:00.374 INFO:teuthology.orchestra.run.smithi047.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T01:41:00.376 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T01:41:00.384 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T01:41:00.389 INFO:teuthology.orchestra.run.smithi047.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T01:41:00.390 INFO:teuthology.orchestra.run.smithi047.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T01:41:00.407 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T01:41:00.421 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T01:41:00.477 INFO:teuthology.orchestra.run.smithi047.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T01:41:00.477 INFO:teuthology.orchestra.run.smithi047.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T01:41:00.477 INFO:teuthology.orchestra.run.smithi047.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T01:41:00.478 INFO:teuthology.orchestra.run.smithi047.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T01:41:00.478 INFO:teuthology.orchestra.run.smithi047.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T01:41:00.480 INFO:teuthology.orchestra.run.smithi047.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T01:41:00.482 INFO:teuthology.orchestra.run.smithi047.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T01:41:00.484 INFO:teuthology.orchestra.run.smithi047.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T01:41:00.489 INFO:teuthology.orchestra.run.smithi047.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T01:41:00.496 INFO:teuthology.orchestra.run.smithi047.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T01:41:00.496 INFO:teuthology.orchestra.run.smithi047.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T01:41:00.497 INFO:teuthology.orchestra.run.smithi047.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T01:41:00.498 INFO:teuthology.orchestra.run.smithi047.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T01:41:00.498 INFO:teuthology.orchestra.run.smithi047.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T01:41:00.511 INFO:teuthology.orchestra.run.smithi047.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T01:41:00.511 INFO:teuthology.orchestra.run.smithi047.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T01:41:00.513 INFO:teuthology.orchestra.run.smithi047.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T01:41:00.514 INFO:teuthology.orchestra.run.smithi047.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T01:41:00.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T01:41:00.551 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T01:41:00.581 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T01:41:00.611 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T01:41:00.618 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T01:41:00.661 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T01:41:00.675 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T01:41:00.721 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T01:41:00.729 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T01:41:00.738 INFO:teuthology.orchestra.run.smithi047.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T01:41:00.740 INFO:teuthology.orchestra.run.smithi047.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T01:41:00.751 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T01:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T01:41:00.758 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T01:41:00.760 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T01:41:00.766 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T01:41:00.767 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T01:41:00.768 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T01:41:00.769 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T01:41:00.801 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T01:41:00.803 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T01:41:00.806 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T01:41:00.835 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T01:41:00.855 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T01:41:00.856 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T01:41:00.857 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T01:41:00.858 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T01:41:00.859 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T01:41:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T01:41:00.860 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T01:41:00.862 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T01:41:00.889 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T01:41:00.945 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T01:41:00.947 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T01:41:00.947 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T01:41:00.955 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T01:41:00.955 INFO:teuthology.orchestra.run.smithi155.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T01:41:00.966 INFO:teuthology.orchestra.run.smithi155.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T01:41:00.966 INFO:teuthology.orchestra.run.smithi155.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T01:41:00.967 INFO:teuthology.orchestra.run.smithi155.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T01:41:00.970 INFO:teuthology.orchestra.run.smithi155.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T01:41:00.976 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T01:41:00.984 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T01:41:01.009 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T01:41:01.016 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T01:41:01.017 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T01:41:01.023 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T01:41:01.024 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T01:41:01.024 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T01:41:01.025 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T01:41:01.047 INFO:teuthology.orchestra.run.smithi155.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T01:41:01.047 INFO:teuthology.orchestra.run.smithi155.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T01:41:01.051 INFO:teuthology.orchestra.run.smithi155.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T01:41:01.053 INFO:teuthology.orchestra.run.smithi155.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T01:41:01.055 INFO:teuthology.orchestra.run.smithi155.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T01:41:01.058 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T01:41:01.111 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T01:41:01.112 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T01:41:01.113 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T01:41:01.114 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T01:41:01.116 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T01:41:01.117 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T01:41:01.118 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T01:41:01.120 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T01:41:01.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T01:41:01.140 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T01:41:01.190 INFO:teuthology.orchestra.run.smithi155.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T01:41:01.203 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T01:41:01.206 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T01:41:01.207 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T01:41:01.215 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T01:41:01.216 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T01:41:01.217 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T01:41:01.218 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T01:41:01.222 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T01:41:01.224 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T01:41:01.257 INFO:teuthology.orchestra.run.smithi155.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T01:41:01.303 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T01:41:01.304 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T01:41:01.307 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T01:41:01.309 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T01:41:01.311 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T01:41:01.312 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T01:41:01.313 INFO:teuthology.orchestra.run.smithi033.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T01:41:01.342 INFO:teuthology.orchestra.run.smithi155.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T01:41:01.345 INFO:teuthology.orchestra.run.smithi155.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T01:41:01.347 INFO:teuthology.orchestra.run.smithi155.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T01:41:01.488 INFO:teuthology.orchestra.run.smithi155.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T01:41:01.492 INFO:teuthology.orchestra.run.smithi155.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T01:41:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T01:41:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T01:41:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T01:41:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T01:41:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T01:41:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T01:41:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T01:41:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T01:41:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T01:41:01.623 INFO:teuthology.orchestra.run.smithi155.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T01:41:01.638 INFO:teuthology.orchestra.run.smithi033.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T01:41:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T01:41:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T01:41:01.667 INFO:teuthology.orchestra.run.smithi033.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T01:41:01.714 INFO:teuthology.orchestra.run.smithi033.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T01:41:01.716 INFO:teuthology.orchestra.run.smithi047.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T01:41:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T01:41:01.737 INFO:teuthology.orchestra.run.smithi033.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T01:41:01.737 INFO:teuthology.orchestra.run.smithi033.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T01:41:01.737 INFO:teuthology.orchestra.run.smithi033.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T01:41:01.738 INFO:teuthology.orchestra.run.smithi033.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T01:41:01.738 INFO:teuthology.orchestra.run.smithi033.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T01:41:01.739 INFO:teuthology.orchestra.run.smithi033.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T01:41:01.740 INFO:teuthology.orchestra.run.smithi155.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T01:41:01.740 INFO:teuthology.orchestra.run.smithi033.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T01:41:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T01:41:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T01:41:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T01:41:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T01:41:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T01:41:01.803 INFO:teuthology.orchestra.run.smithi033.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T01:41:01.804 INFO:teuthology.orchestra.run.smithi033.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T01:41:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T01:41:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T01:41:01.870 INFO:teuthology.orchestra.run.smithi047.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T01:41:01.874 INFO:teuthology.orchestra.run.smithi047.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T01:41:01.877 INFO:teuthology.orchestra.run.smithi047.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T01:41:01.885 INFO:teuthology.orchestra.run.smithi033.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T01:41:01.890 INFO:teuthology.orchestra.run.smithi033.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T01:41:01.893 INFO:teuthology.orchestra.run.smithi033.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T01:41:01.897 INFO:teuthology.orchestra.run.smithi033.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T01:41:01.898 INFO:teuthology.orchestra.run.smithi033.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T01:41:01.901 INFO:teuthology.orchestra.run.smithi033.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T01:41:01.903 INFO:teuthology.orchestra.run.smithi033.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T01:41:01.905 INFO:teuthology.orchestra.run.smithi047.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T01:41:01.967 INFO:teuthology.orchestra.run.smithi033.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T01:41:01.977 INFO:teuthology.orchestra.run.smithi033.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T01:41:01.981 INFO:teuthology.orchestra.run.smithi033.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T01:41:01.993 INFO:teuthology.orchestra.run.smithi155.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T01:41:01.994 INFO:teuthology.orchestra.run.smithi155.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T01:41:01.994 INFO:teuthology.orchestra.run.smithi155.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T01:41:01.994 INFO:teuthology.orchestra.run.smithi155.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T01:41:01.995 INFO:teuthology.orchestra.run.smithi155.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T01:41:01.996 INFO:teuthology.orchestra.run.smithi155.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T01:41:02.008 INFO:teuthology.orchestra.run.smithi155.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T01:41:02.009 INFO:teuthology.orchestra.run.smithi155.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T01:41:02.043 INFO:teuthology.orchestra.run.smithi155.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T01:41:02.046 INFO:teuthology.orchestra.run.smithi047.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T01:41:02.075 INFO:teuthology.orchestra.run.smithi155.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T01:41:02.142 INFO:teuthology.orchestra.run.smithi033.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T01:41:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T01:41:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T01:41:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T01:41:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T01:41:02.145 INFO:teuthology.orchestra.run.smithi033.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T01:41:02.156 INFO:teuthology.orchestra.run.smithi155.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T01:41:02.157 INFO:teuthology.orchestra.run.smithi155.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T01:41:02.157 INFO:teuthology.orchestra.run.smithi155.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T01:41:02.158 INFO:teuthology.orchestra.run.smithi033.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T01:41:02.159 INFO:teuthology.orchestra.run.smithi033.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T01:41:02.159 INFO:teuthology.orchestra.run.smithi033.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T01:41:02.161 INFO:teuthology.orchestra.run.smithi155.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T01:41:02.163 INFO:teuthology.orchestra.run.smithi155.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T01:41:02.224 INFO:teuthology.orchestra.run.smithi033.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T01:41:02.306 INFO:teuthology.orchestra.run.smithi033.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T01:41:02.306 INFO:teuthology.orchestra.run.smithi033.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T01:41:02.308 INFO:teuthology.orchestra.run.smithi033.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T01:41:02.311 INFO:teuthology.orchestra.run.smithi033.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T01:41:02.312 INFO:teuthology.orchestra.run.smithi033.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T01:41:02.542 INFO:teuthology.orchestra.run.smithi155.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T01:41:02.542 INFO:teuthology.orchestra.run.smithi155.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T01:41:02.555 INFO:teuthology.orchestra.run.smithi155.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T01:41:02.607 INFO:teuthology.orchestra.run.smithi155.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T01:41:02.608 INFO:teuthology.orchestra.run.smithi155.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T01:41:02.610 INFO:teuthology.orchestra.run.smithi155.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T01:41:02.669 INFO:teuthology.orchestra.run.smithi033.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T01:41:02.805 INFO:teuthology.orchestra.run.smithi033.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T01:41:02.815 INFO:teuthology.orchestra.run.smithi033.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T01:41:02.820 INFO:teuthology.orchestra.run.smithi033.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T01:41:02.848 INFO:teuthology.orchestra.run.smithi155.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T01:41:02.876 INFO:teuthology.orchestra.run.smithi033.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T01:41:02.878 INFO:teuthology.orchestra.run.smithi033.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T01:41:02.890 INFO:teuthology.orchestra.run.smithi047.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T01:41:02.890 INFO:teuthology.orchestra.run.smithi047.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T01:41:02.900 INFO:teuthology.orchestra.run.smithi047.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T01:41:02.923 INFO:teuthology.orchestra.run.smithi033.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T01:41:02.948 INFO:teuthology.orchestra.run.smithi047.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T01:41:02.950 INFO:teuthology.orchestra.run.smithi047.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T01:41:02.952 INFO:teuthology.orchestra.run.smithi047.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T01:41:03.176 INFO:teuthology.orchestra.run.smithi155.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T01:41:03.176 INFO:teuthology.orchestra.run.smithi155.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T01:41:03.180 INFO:teuthology.orchestra.run.smithi047.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T01:41:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T01:41:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T01:41:03.585 INFO:teuthology.orchestra.run.smithi033.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T01:41:03.587 INFO:teuthology.orchestra.run.smithi033.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T01:41:03.605 INFO:teuthology.orchestra.run.smithi033.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T01:41:05.333 INFO:teuthology.orchestra.run.smithi033.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T01:41:05.545 INFO:teuthology.orchestra.run.smithi033.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T01:41:05.548 INFO:teuthology.orchestra.run.smithi033.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T01:41:05.551 INFO:teuthology.orchestra.run.smithi033.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T01:41:05.627 INFO:teuthology.orchestra.run.smithi033.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T01:41:05.633 INFO:teuthology.orchestra.run.smithi155.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T01:41:05.638 INFO:teuthology.orchestra.run.smithi155.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T01:41:05.638 INFO:teuthology.orchestra.run.smithi155.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T01:41:05.857 INFO:teuthology.orchestra.run.smithi033.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T01:41:06.225 INFO:teuthology.orchestra.run.smithi047.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T01:41:06.230 INFO:teuthology.orchestra.run.smithi047.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T01:41:06.231 INFO:teuthology.orchestra.run.smithi047.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T01:41:06.296 INFO:teuthology.orchestra.run.smithi155.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T01:41:06.771 INFO:teuthology.orchestra.run.smithi047.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T01:41:06.799 INFO:teuthology.orchestra.run.smithi033.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T01:41:06.800 INFO:teuthology.orchestra.run.smithi033.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T01:41:06.813 INFO:teuthology.orchestra.run.smithi033.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T01:41:06.830 INFO:teuthology.orchestra.run.smithi033.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T01:41:06.831 INFO:teuthology.orchestra.run.smithi033.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T01:41:06.832 INFO:teuthology.orchestra.run.smithi033.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T01:41:06.949 INFO:teuthology.orchestra.run.smithi033.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T01:41:06.950 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 201 MB in 7s (30.8 MB/s) 2024-09-15T01:41:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T01:41:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T01:41:07.127 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T01:41:07.186 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T01:41:07.190 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T01:41:07.275 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:41:07.431 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 201 MB in 7s (28.9 MB/s) 2024-09-15T01:41:07.581 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T01:41:07.639 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T01:41:07.643 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T01:41:07.653 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:07.763 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:41:07.779 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:08.141 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:08.180 INFO:teuthology.orchestra.run.smithi033.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T01:41:08.185 INFO:teuthology.orchestra.run.smithi033.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T01:41:08.186 INFO:teuthology.orchestra.run.smithi033.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T01:41:08.268 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:08.458 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:08.546 INFO:teuthology.orchestra.run.smithi033.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T01:41:08.567 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:08.988 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:09.113 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:09.205 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 201 MB in 9s (23.6 MB/s) 2024-09-15T01:41:09.221 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T01:41:09.241 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:09.288 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:09.375 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T01:41:09.432 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T01:41:09.436 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T01:41:09.515 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:41:09.632 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rados. 2024-09-15T01:41:09.648 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:09.691 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:09.801 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T01:41:09.817 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:09.883 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:09.894 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:10.019 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:10.069 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T01:41:10.084 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:10.119 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:10.245 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rados. 2024-09-15T01:41:10.265 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:10.303 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:10.371 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T01:41:10.387 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:10.429 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:10.665 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T01:41:10.680 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:10.722 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:10.781 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T01:41:10.799 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T01:41:10.815 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:10.857 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.000 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T01:41:11.018 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:11.058 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.178 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T01:41:11.195 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T01:41:11.261 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T01:41:11.420 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T01:41:11.436 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:11.468 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T01:41:11.478 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.486 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:11.514 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T01:41:11.527 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.531 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:11.572 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.814 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T01:41:11.834 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T01:41:11.871 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rados. 2024-09-15T01:41:11.872 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T01:41:11.890 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:11.929 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:11.942 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T01:41:11.961 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T01:41:11.993 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:41:12.125 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T01:41:12.145 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:12.183 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:12.273 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T01:41:12.293 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:12.331 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:12.345 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T01:41:12.365 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T01:41:12.403 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:41:12.567 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T01:41:12.570 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T01:41:12.586 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T01:41:12.617 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:12.628 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:41:12.765 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T01:41:12.780 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T01:41:12.823 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:41:12.945 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T01:41:12.965 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:13.003 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:13.006 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T01:41:13.023 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T01:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:41:13.226 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T01:41:13.243 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T01:41:13.284 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:41:13.331 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T01:41:13.342 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T01:41:13.347 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T01:41:13.358 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T01:41:13.389 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T01:41:13.417 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:41:13.587 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T01:41:13.606 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T01:41:13.642 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T01:41:13.645 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:41:13.658 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:13.700 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:13.803 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T01:41:13.822 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T01:41:13.885 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:41:14.053 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T01:41:14.070 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T01:41:14.111 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:41:14.124 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T01:41:14.144 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:14.193 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:14.231 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T01:41:14.247 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T01:41:14.306 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:41:14.481 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T01:41:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T01:41:14.539 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:41:14.695 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T01:41:14.714 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:14.726 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T01:41:14.742 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:14.752 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:14.803 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:15.050 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T01:41:15.067 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T01:41:15.117 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:41:15.340 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T01:41:15.358 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:15.372 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T01:41:15.389 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:15.390 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:15.478 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T01:41:15.495 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T01:41:15.527 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:15.536 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:41:15.818 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T01:41:15.837 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T01:41:15.876 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:41:15.889 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T01:41:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T01:41:15.907 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T01:41:15.923 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:15.947 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:41:15.965 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:16.237 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librgw2. 2024-09-15T01:41:16.254 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:16.329 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:16.360 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T01:41:16.376 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T01:41:16.409 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T01:41:16.418 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:41:16.429 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:16.478 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:16.788 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librgw2. 2024-09-15T01:41:16.806 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:16.871 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:16.962 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T01:41:16.979 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:17.020 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:17.082 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T01:41:17.101 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:17.132 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:17.415 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T01:41:17.431 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T01:41:17.473 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:41:17.476 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liboath0. 2024-09-15T01:41:17.497 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T01:41:17.553 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T01:41:17.674 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T01:41:17.690 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:17.732 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:17.876 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T01:41:17.879 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T01:41:17.892 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T01:41:17.895 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:17.934 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:41:17.937 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:18.060 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liboath0. 2024-09-15T01:41:18.076 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T01:41:18.113 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T01:41:18.263 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librgw2. 2024-09-15T01:41:18.281 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:18.321 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:18.358 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-common. 2024-09-15T01:41:18.377 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:18.408 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:18.437 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T01:41:18.453 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:18.496 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:18.982 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-common. 2024-09-15T01:41:18.999 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:19.074 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:19.107 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T01:41:19.124 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:19.164 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:19.526 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liboath0. 2024-09-15T01:41:19.542 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T01:41:19.584 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T01:41:19.878 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T01:41:19.897 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:19.933 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:20.482 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-common. 2024-09-15T01:41:20.498 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:20.531 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:20.963 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-base. 2024-09-15T01:41:20.982 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:21.032 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-base. 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:21.648 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T01:41:21.968 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:41:22.355 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T01:41:22.371 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T01:41:22.534 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T01:41:22.554 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T01:41:22.800 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T01:41:23.006 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T01:41:23.023 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T01:41:23.036 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T01:41:23.052 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:23.053 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-base. 2024-09-15T01:41:23.064 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T01:41:23.073 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:23.094 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:23.122 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:23.400 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T01:41:23.416 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T01:41:23.483 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T01:41:23.508 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T01:41:23.525 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T01:41:23.566 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:41:23.736 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T01:41:23.754 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:23.828 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:23.894 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T01:41:23.911 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T01:41:23.952 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:41:23.966 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T01:41:23.982 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T01:41:24.016 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:41:24.233 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T01:41:24.249 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T01:41:24.291 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:41:24.297 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T01:41:24.314 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T01:41:24.356 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T01:41:24.403 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T01:41:24.419 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T01:41:24.453 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T01:41:24.619 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T01:41:24.635 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T01:41:24.677 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:41:24.693 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T01:41:24.711 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T01:41:24.713 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T01:41:24.730 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T01:41:24.750 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:41:24.763 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T01:41:25.025 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T01:41:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:25.055 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T01:41:25.072 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T01:41:25.077 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T01:41:25.091 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:25.094 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T01:41:25.113 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T01:41:25.137 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T01:41:25.442 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T01:41:25.448 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T01:41:25.459 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T01:41:25.462 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T01:41:25.469 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T01:41:25.479 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T01:41:25.500 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:41:25.507 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:41:25.521 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:41:25.810 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-mako. 2024-09-15T01:41:25.823 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T01:41:25.829 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T01:41:25.840 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T01:41:25.845 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T01:41:25.861 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T01:41:25.868 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:41:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:41:25.911 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T01:41:26.197 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T01:41:26.212 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T01:41:26.219 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T01:41:26.222 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T01:41:26.235 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T01:41:26.242 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T01:41:26.254 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:41:26.277 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T01:41:26.280 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:41:26.566 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T01:41:26.585 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T01:41:26.588 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T01:41:26.604 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T01:41:26.608 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-mako. 2024-09-15T01:41:26.624 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:41:26.626 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T01:41:26.638 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:41:26.666 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:41:26.902 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webob. 2024-09-15T01:41:26.921 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T01:41:26.941 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T01:41:26.958 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T01:41:26.993 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:41:26.999 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T01:41:27.029 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T01:41:27.047 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T01:41:27.078 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:41:27.285 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T01:41:27.305 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T01:41:27.322 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T01:41:27.338 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T01:41:27.343 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:41:27.380 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:41:27.406 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T01:41:27.423 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T01:41:27.464 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:41:27.630 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-mako. 2024-09-15T01:41:27.646 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T01:41:27.667 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T01:41:27.683 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T01:41:27.687 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:41:27.725 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T01:41:27.809 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webob. 2024-09-15T01:41:27.825 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T01:41:27.867 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:41:28.032 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T01:41:28.049 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T01:41:28.061 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T01:41:28.078 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T01:41:28.091 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:41:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T01:41:28.252 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T01:41:28.295 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:41:28.377 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T01:41:28.397 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T01:41:28.435 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:41:28.456 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T01:41:28.474 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T01:41:28.514 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T01:41:28.637 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T01:41:28.653 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T01:41:28.695 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T01:41:28.721 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webob. 2024-09-15T01:41:28.741 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T01:41:28.779 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:41:28.843 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-paste. 2024-09-15T01:41:28.860 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T01:41:28.901 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:41:29.057 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T01:41:29.075 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T01:41:29.099 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T01:41:29.115 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T01:41:29.135 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:41:29.157 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:41:29.222 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T01:41:29.239 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T01:41:29.280 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:41:29.485 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T01:41:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T01:41:29.503 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T01:41:29.519 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T01:41:29.543 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T01:41:29.560 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T01:41:29.602 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T01:41:29.619 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T01:41:29.660 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:41:29.879 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-paste. 2024-09-15T01:41:29.898 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T01:41:29.905 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T01:41:29.921 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T01:41:29.937 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:41:29.940 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T01:41:29.958 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T01:41:29.966 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:41:30.000 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T01:41:30.283 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T01:41:30.299 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T01:41:30.337 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T01:41:30.340 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:41:30.357 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T01:41:30.395 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:41:30.669 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T01:41:30.685 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T01:41:30.727 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:41:31.030 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T01:41:31.047 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T01:41:31.088 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T01:41:31.391 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-paste. 2024-09-15T01:41:31.408 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T01:41:31.449 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:41:31.616 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T01:41:31.632 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T01:41:31.666 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:41:31.778 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T01:41:31.794 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T01:41:31.835 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:41:31.847 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T01:41:31.864 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T01:41:31.905 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T01:41:32.023 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T01:41:32.042 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:32.095 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:32.138 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T01:41:32.155 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T01:41:32.197 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:41:32.218 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T01:41:32.238 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T01:41:32.277 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:41:32.423 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T01:41:32.442 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:32.481 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:32.525 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T01:41:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T01:41:32.583 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T01:41:32.642 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T01:41:32.662 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T01:41:32.700 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:41:32.886 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T01:41:32.903 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T01:41:32.944 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:41:32.993 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T01:41:33.013 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:33.051 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:33.257 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T01:41:33.261 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T01:41:33.277 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T01:41:33.279 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:33.315 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:41:33.318 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:33.689 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T01:41:33.706 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T01:41:33.707 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:33.721 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:33.747 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:33.765 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:33.805 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T01:41:33.823 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:33.879 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:34.125 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T01:41:34.142 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:34.183 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:34.283 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T01:41:34.300 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:34.361 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:34.610 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T01:41:34.629 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:34.660 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:35.328 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T01:41:35.345 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:35.378 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:35.422 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T01:41:35.441 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:36.009 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph. 2024-09-15T01:41:36.027 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:36.067 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:36.395 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T01:41:36.413 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:36.469 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:36.865 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T01:41:36.882 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:36.914 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:37.409 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephadm. 2024-09-15T01:41:37.426 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:37.476 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:37.524 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph. 2024-09-15T01:41:37.540 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:37.582 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:37.625 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph. 2024-09-15T01:41:37.642 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:37.682 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:37.821 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T01:41:37.837 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T01:41:37.879 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:41:37.982 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T01:41:37.997 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:38.027 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:38.051 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T01:41:38.071 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:38.126 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:38.182 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T01:41:38.197 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:38.240 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:38.447 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T01:41:38.464 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:38.505 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:38.546 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T01:41:38.562 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:38.595 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:38.602 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T01:41:38.621 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T01:41:38.660 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T01:41:38.938 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T01:41:38.955 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T01:41:38.996 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:41:39.042 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephadm. 2024-09-15T01:41:39.058 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:39.100 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:39.149 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package cephadm. 2024-09-15T01:41:39.165 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:39.226 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:39.265 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T01:41:39.281 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T01:41:39.323 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:41:39.413 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T01:41:39.430 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T01:41:39.471 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:41:39.535 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T01:41:39.554 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T01:41:39.612 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:41:39.618 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-routes. 2024-09-15T01:41:39.637 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T01:41:39.676 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T01:41:39.799 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T01:41:39.815 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:39.857 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:39.962 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T01:41:39.971 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T01:41:39.979 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:39.988 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:40.030 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:40.219 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T01:41:40.235 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T01:41:40.276 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T01:41:40.407 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T01:41:40.426 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T01:41:40.465 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T01:41:40.529 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T01:41:40.547 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T01:41:40.587 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:41:40.751 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T01:41:40.770 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T01:41:40.809 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:41:40.856 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T01:41:40.872 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T01:41:40.914 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:41:41.078 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T01:41:41.095 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T01:41:41.153 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:41:41.251 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-routes. 2024-09-15T01:41:41.270 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T01:41:41.309 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T01:41:41.337 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T01:41:41.355 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T01:41:41.428 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:41:41.514 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-routes. 2024-09-15T01:41:41.531 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T01:41:41.572 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T01:41:41.645 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T01:41:41.663 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:41.703 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:41.781 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T01:41:41.797 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T01:41:41.839 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:41:41.917 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T01:41:41.933 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:41.992 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:42.184 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T01:41:42.202 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T01:41:42.242 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T01:41:43.009 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T01:41:43.025 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T01:41:43.067 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:41:43.316 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T01:41:43.332 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T01:41:43.394 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T01:41:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:41:43.411 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T01:41:43.453 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:41:43.593 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T01:41:43.611 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T01:41:43.651 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:43.727 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T01:41:43.740 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T01:41:43.743 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T01:41:43.760 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T01:41:43.784 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:41:43.790 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T01:41:44.112 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T01:41:44.131 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T01:41:44.170 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T01:41:44.222 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T01:41:44.239 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T01:41:44.279 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T01:41:44.683 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T01:41:44.699 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T01:41:44.741 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:45.132 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T01:41:45.151 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T01:41:45.190 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:45.442 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T01:41:45.462 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:45.500 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:45.546 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T01:41:45.563 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T01:41:45.604 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:45.752 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T01:41:45.768 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T01:41:45.810 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T01:41:46.150 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T01:41:46.166 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T01:41:46.205 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T01:41:46.207 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T01:41:46.223 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T01:41:46.263 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:46.652 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T01:41:46.671 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T01:41:46.719 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:41:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T01:41:46.972 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:46.978 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T01:41:46.998 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T01:41:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:47.061 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T01:41:47.339 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T01:41:47.356 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T01:41:47.398 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:41:47.752 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T01:41:47.768 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T01:41:47.810 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:41:47.869 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T01:41:47.890 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:47.927 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:48.122 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T01:41:48.139 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T01:41:48.175 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T01:41:48.179 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T01:41:48.194 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T01:41:48.232 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T01:41:48.525 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T01:41:48.544 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T01:41:48.583 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T01:41:48.644 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T01:41:48.660 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T01:41:48.704 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:41:48.928 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T01:41:48.946 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T01:41:48.986 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:41:49.013 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T01:41:49.031 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T01:41:49.071 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:41:49.122 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T01:41:49.140 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T01:41:49.180 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T01:41:49.282 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T01:41:49.299 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T01:41:49.339 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:41:49.424 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T01:41:49.440 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T01:41:49.482 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T01:41:49.500 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T01:41:49.516 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T01:41:49.558 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:41:49.693 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T01:41:49.713 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T01:41:49.752 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:41:49.777 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T01:41:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T01:41:49.835 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T01:41:49.920 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T01:41:49.936 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T01:41:49.969 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:41:50.046 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T01:41:50.063 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T01:41:50.104 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:41:50.155 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T01:41:50.171 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T01:41:50.213 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:41:50.322 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T01:41:50.342 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T01:41:50.406 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T01:41:50.445 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T01:41:50.463 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T01:41:50.492 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T01:41:50.503 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:41:50.510 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T01:41:50.541 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:41:50.743 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T01:41:50.760 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T01:41:50.801 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T01:41:50.903 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T01:41:50.920 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T01:41:50.961 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:41:51.127 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T01:41:51.146 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T01:41:51.146 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:51.162 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T01:41:51.193 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:51.203 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:41:51.281 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T01:41:51.297 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T01:41:51.339 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:41:51.522 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T01:41:51.523 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T01:41:51.539 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T01:41:51.542 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T01:41:51.580 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:41:51.581 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:41:51.918 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T01:41:51.927 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T01:41:51.934 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T01:41:51.935 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T01:41:51.944 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T01:41:51.951 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:51.976 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:41:51.985 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:41:52.017 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:52.287 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package jq. 2024-09-15T01:41:52.306 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:52.330 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T01:41:52.337 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:52.348 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T01:41:52.388 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:41:52.590 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package socat. 2024-09-15T01:41:52.607 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T01:41:52.658 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T01:41:52.673 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T01:41:52.678 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:52.716 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:52.724 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T01:41:52.741 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T01:41:52.782 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:41:52.973 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T01:41:52.989 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T01:41:53.022 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T01:41:53.031 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:41:53.039 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T01:41:53.080 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:41:53.421 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-test. 2024-09-15T01:41:53.433 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T01:41:53.439 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:53.451 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:53.477 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T01:41:53.479 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:53.492 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:53.495 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:53.535 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package jq. 2024-09-15T01:41:53.737 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:53.778 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:53.871 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T01:41:53.888 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T01:41:53.929 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:41:54.064 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package socat. 2024-09-15T01:41:54.082 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T01:41:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T01:41:54.299 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T01:41:54.317 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:54.357 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:54.434 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T01:41:54.454 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T01:41:54.491 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:41:54.653 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package jq. 2024-09-15T01:41:54.669 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T01:41:54.711 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:41:54.831 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-test. 2024-09-15T01:41:54.850 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:54.889 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:54.997 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package socat. 2024-09-15T01:41:55.013 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T01:41:55.055 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T01:41:55.389 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T01:41:55.407 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T01:41:55.447 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:41:55.844 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-test. 2024-09-15T01:41:55.861 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:55.902 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:58.052 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T01:41:58.071 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:58.110 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:58.457 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T01:41:58.476 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:41:58.515 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:41:59.027 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T01:41:59.045 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:41:59.088 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:59.446 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T01:41:59.463 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:41:59.481 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T01:41:59.497 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T01:41:59.512 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:41:59.565 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:41:59.819 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T01:41:59.836 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:41:59.894 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T01:41:59.902 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:41:59.911 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:41:59.943 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:00.390 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T01:42:00.407 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:00.448 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:00.556 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T01:42:00.573 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T01:42:00.636 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:00.809 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T01:42:00.826 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T01:42:00.877 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:42:00.962 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T01:42:00.980 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:42:01.053 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:01.197 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T01:42:01.216 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:42:01.255 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:01.565 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T01:42:01.583 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T01:42:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T01:42:01.623 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:01.642 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:01.962 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T01:42:01.982 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T01:42:02.019 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:02.060 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T01:42:02.077 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T01:42:02.135 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:42:02.339 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-attr. 2024-09-15T01:42:02.356 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T01:42:02.397 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T01:42:02.481 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T01:42:02.498 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T01:42:02.556 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:02.725 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T01:42:02.742 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T01:42:02.783 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T01:42:02.903 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T01:42:02.964 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:03.119 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T01:42:03.136 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T01:42:03.177 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:03.293 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T01:42:03.312 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T01:42:03.351 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:03.522 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T01:42:03.542 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T01:42:03.580 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:03.637 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-attr. 2024-09-15T01:42:03.655 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T01:42:03.695 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T01:42:03.989 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T01:42:04.010 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T01:42:04.055 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:04.067 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T01:42:04.084 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T01:42:04.158 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:04.231 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T01:42:04.252 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T01:42:04.314 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:04.367 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T01:42:04.384 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T01:42:04.450 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:04.462 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T01:42:04.478 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T01:42:04.519 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:04.675 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T01:42:04.693 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T01:42:04.733 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:04.769 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T01:42:04.790 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T01:42:04.814 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T01:42:04.827 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:04.834 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T01:42:04.872 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:05.061 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-attr. 2024-09-15T01:42:05.078 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T01:42:05.119 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T01:42:05.175 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-py. 2024-09-15T01:42:05.195 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T01:42:05.233 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:05.272 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T01:42:05.291 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T01:42:05.332 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:05.480 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T01:42:05.497 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T01:42:05.538 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T01:42:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T01:42:05.628 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:05.651 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T01:42:05.668 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T01:42:05.709 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:05.939 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T01:42:05.956 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T01:42:05.966 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T01:42:05.987 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T01:42:06.013 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:06.016 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:06.028 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T01:42:06.045 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T01:42:06.078 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:06.342 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T01:42:06.363 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T01:42:06.385 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T01:42:06.400 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T01:42:06.402 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T01:42:06.477 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:06.498 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-py. 2024-09-15T01:42:06.518 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T01:42:06.555 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:06.773 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T01:42:06.790 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T01:42:06.831 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:06.869 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T01:42:06.890 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T01:42:06.927 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:06.947 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T01:42:06.965 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T01:42:07.005 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:07.117 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T01:42:07.135 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T01:42:07.175 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:07.238 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T01:42:07.256 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T01:42:07.296 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:07.342 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T01:42:07.362 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T01:42:07.391 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:07.625 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T01:42:07.643 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T01:42:07.682 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T01:42:07.728 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T01:42:07.746 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T01:42:07.786 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:07.962 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package radosgw. 2024-09-15T01:42:07.980 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:08.020 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:08.020 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T01:42:08.037 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T01:42:08.078 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:08.189 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-py. 2024-09-15T01:42:08.206 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T01:42:08.247 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:08.348 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T01:42:08.366 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T01:42:08.405 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:08.592 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T01:42:08.609 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T01:42:08.675 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:08.986 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T01:42:09.007 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T01:42:09.052 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:09.152 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package radosgw. 2024-09-15T01:42:09.171 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:09.210 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:09.397 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T01:42:09.416 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T01:42:09.455 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T01:42:09.765 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T01:42:09.783 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:09.815 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:09.827 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T01:42:09.847 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T01:42:09.894 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:10.117 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T01:42:10.197 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T01:42:10.216 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T01:42:10.247 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:10.406 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:42:10.515 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:42:10.624 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:42:10.817 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:10.966 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T01:42:10.984 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:11.024 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:11.050 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package radosgw. 2024-09-15T01:42:11.069 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:11.108 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:11.336 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T01:42:11.386 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user cephadm....done 2024-09-15T01:42:11.541 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:11.629 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:42:11.701 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:42:11.755 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:42:11.881 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:42:12.035 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T01:42:12.091 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:12.311 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:12.462 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T01:42:12.589 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:12.689 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user cephadm....done 2024-09-15T01:42:12.850 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:12.932 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T01:42:12.985 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:42:12.987 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T01:42:13.005 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T01:42:13.045 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:13.328 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T01:42:13.342 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:42:13.371 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T01:42:13.627 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T01:42:13.629 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:13.684 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T01:42:13.756 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T01:42:13.786 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T01:42:13.882 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:13.887 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:42:13.888 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T01:42:14.105 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:14.183 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T01:42:14.187 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:14.480 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:14.612 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:42:14.675 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user cephadm....done 2024-09-15T01:42:14.773 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:14.820 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:14.896 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T01:42:14.920 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T01:42:14.932 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:42:15.173 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:42:15.208 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:15.273 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T01:42:15.440 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:15.501 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:42:15.583 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T01:42:15.602 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:15.709 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T01:42:15.725 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:15.834 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T01:42:15.962 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T01:42:16.018 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:16.152 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:42:16.169 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T01:42:16.279 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:16.420 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:16.547 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T01:42:16.555 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:16.687 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:42:16.813 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:16.839 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T01:42:16.932 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:42:17.132 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T01:42:17.181 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T01:42:17.225 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:42:17.442 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T01:42:17.466 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:17.720 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:17.776 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:17.935 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T01:42:18.060 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:42:18.248 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:18.360 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T01:42:18.361 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:42:18.628 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:42:18.645 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T01:42:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T01:42:18.737 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:18.863 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:18.888 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T01:42:19.005 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T01:42:19.156 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:42:19.197 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T01:42:19.315 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:42:19.365 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T01:42:19.441 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:19.567 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:19.575 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:42:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T01:42:19.885 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:42:19.951 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:42:20.059 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T01:42:20.303 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:42:20.311 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T01:42:20.454 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:20.468 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T01:42:20.663 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:42:20.764 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:20.828 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T01:42:20.873 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:21.023 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T01:42:21.096 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T01:42:21.182 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:21.199 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:42:21.325 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:42:21.451 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:21.526 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:21.563 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T01:42:21.660 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:21.736 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:21.865 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T01:42:22.020 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:42:22.045 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:22.137 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:42:22.166 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T01:42:22.171 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:42:22.263 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:22.306 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:42:22.316 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T01:42:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T01:42:22.608 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:42:22.727 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T01:42:22.781 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:22.875 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T01:42:23.104 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T01:42:23.116 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:23.210 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:42:23.242 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:42:23.336 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:42:23.376 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:42:23.447 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T01:42:23.612 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T01:42:23.720 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:42:23.815 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T01:42:23.905 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:42:23.941 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:24.154 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T01:42:24.206 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:42:24.401 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:24.455 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:42:24.552 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T01:42:24.572 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T01:42:24.581 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:42:24.825 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:42:24.882 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T01:42:24.887 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T01:42:24.951 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:42:24.988 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T01:42:25.097 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T01:42:25.192 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:42:25.277 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:42:25.507 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T01:42:25.509 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:42:25.629 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:42:25.755 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:42:25.858 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:25.881 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:42:25.922 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T01:42:26.009 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T01:42:26.135 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T01:42:26.190 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:42:26.212 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:42:26.338 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:42:26.458 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:26.486 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T01:42:26.584 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T01:42:26.664 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:42:26.771 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T01:42:27.024 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:42:27.030 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T01:42:27.159 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:42:27.189 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T01:42:27.426 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:42:27.507 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T01:42:27.761 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:42:27.808 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T01:42:28.054 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:28.073 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:28.180 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T01:42:28.182 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:28.260 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T01:42:28.300 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T01:42:28.639 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T01:42:28.685 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T01:42:28.912 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T01:42:28.978 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:42:29.047 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T01:42:29.279 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:29.389 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T01:42:29.405 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T01:42:29.694 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:29.766 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T01:42:29.773 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:29.821 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:29.874 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:42:29.892 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T01:42:29.946 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T01:42:30.018 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T01:42:30.176 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:30.302 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:42:30.356 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T01:42:30.379 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T01:42:30.648 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:30.657 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:42:30.670 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:30.773 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T01:42:30.933 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:31.059 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T01:42:31.402 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:31.528 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:42:31.717 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:31.843 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:42:31.872 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:31.969 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:31.998 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:42:32.095 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:32.221 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T01:42:32.324 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:32.337 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:32.446 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:32.538 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:42:32.556 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T01:42:32.664 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T01:42:32.941 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T01:42:32.965 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:33.091 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:33.250 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T01:42:33.298 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:33.298 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:33.396 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:33.551 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T01:42:33.639 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:42:33.677 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T01:42:33.709 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:33.765 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:33.866 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:33.937 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:42:34.001 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T01:42:34.028 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:34.239 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:34.293 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:42:34.304 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T01:42:34.419 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T01:42:34.557 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:42:34.720 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:34.781 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:34.863 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:34.901 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:34.907 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T01:42:35.027 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:35.093 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:35.094 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:35.153 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:35.279 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:35.308 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T01:42:35.430 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T01:42:35.539 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:35.740 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T01:42:35.751 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:42:36.050 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:36.052 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:36.159 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:36.285 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:42:36.379 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:42:36.397 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:36.523 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T01:42:36.649 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T01:42:36.732 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:36.801 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T01:42:36.858 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:36.925 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T01:42:36.984 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.110 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.196 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.227 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T01:42:37.236 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T01:42:37.352 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T01:42:37.538 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T01:42:37.662 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.774 INFO:teuthology.orchestra.run.smithi155.stdout:Adding group ceph....done 2024-09-15T01:42:37.788 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.831 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:37.982 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:38.043 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:38.043 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T01:42:38.108 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:42:38.192 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user ceph....done 2024-09-15T01:42:38.343 INFO:teuthology.orchestra.run.smithi155.stdout:Setting system user ceph properties....done 2024-09-15T01:42:38.364 INFO:teuthology.orchestra.run.smithi155.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T01:42:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:38.506 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T01:42:38.717 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T01:42:38.816 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T01:42:39.010 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T01:42:39.053 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:39.256 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:39.345 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T01:42:39.382 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:39.631 INFO:teuthology.orchestra.run.smithi033.stdout:Adding group ceph....done 2024-09-15T01:42:39.652 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:39.652 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:39.731 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:39.857 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:39.983 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:40.008 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user ceph....done 2024-09-15T01:42:40.134 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:40.175 INFO:teuthology.orchestra.run.smithi033.stdout:Setting system user ceph properties....done 2024-09-15T01:42:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T01:42:40.251 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T01:42:40.276 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:40.347 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T01:42:40.561 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T01:42:40.668 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T01:42:40.689 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T01:42:40.854 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:40.989 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:41.111 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:41.114 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T01:42:41.146 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:41.248 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:41.332 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:41.333 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:41.513 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:41.514 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:41.745 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:41.972 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:41.972 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:42.084 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:42.191 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:42.388 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:42.581 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T01:42:42.687 INFO:teuthology.orchestra.run.smithi047.stdout:Adding group ceph....done 2024-09-15T01:42:42.871 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:42.871 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:43.027 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user ceph....done 2024-09-15T01:42:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:43.289 INFO:teuthology.orchestra.run.smithi047.stdout:Setting system user ceph properties....done 2024-09-15T01:42:43.290 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:43.308 INFO:teuthology.orchestra.run.smithi047.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T01:42:43.416 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T01:42:43.542 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:43.687 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:43.780 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:43.780 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:43.781 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T01:42:43.925 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:43.925 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:44.211 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.219 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.345 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.353 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.371 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.463 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.581 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:44.645 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:44.645 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:42:44.691 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T01:42:44.868 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T01:42:44.876 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:44.876 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:45.161 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T01:42:45.272 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:45.332 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:45.482 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:45.634 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:45.649 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T01:42:45.844 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:45.844 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:46.099 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.285 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.330 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:46.331 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T01:42:46.428 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.554 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.742 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:46.823 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T01:42:46.976 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:46.976 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T01:42:46.992 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T01:42:47.393 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:47.393 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T01:42:47.908 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:47.908 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T01:42:47.996 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T01:42:48.336 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:48.487 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:48.610 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T01:42:48.616 DEBUG:teuthology.parallel:result is None 2024-09-15T01:42:48.638 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:48.870 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:48.870 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T01:42:49.518 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:49.875 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:50.026 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:50.168 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T01:42:50.295 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T01:42:50.321 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T01:42:50.472 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T01:42:50.798 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T01:42:50.933 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T01:42:50.938 DEBUG:teuthology.parallel:result is None 2024-09-15T01:42:53.792 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T01:42:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T01:42:54.382 DEBUG:teuthology.parallel:result is None 2024-09-15T01:42:54.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:42:54.534 DEBUG:teuthology.orchestra.run.smithi033:> dpkg-query -W -f '${Version}' ceph 2024-09-15T01:42:54.557 INFO:teuthology.orchestra.run.smithi033.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.558 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.558 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T01:42:54.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:42:54.703 DEBUG:teuthology.orchestra.run.smithi047:> dpkg-query -W -f '${Version}' ceph 2024-09-15T01:42:54.726 INFO:teuthology.orchestra.run.smithi047.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.726 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.726 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T01:42:54.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:42:54.874 DEBUG:teuthology.orchestra.run.smithi155:> dpkg-query -W -f '${Version}' ceph 2024-09-15T01:42:54.898 INFO:teuthology.orchestra.run.smithi155.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.898 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T01:42:54.898 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T01:42:54.900 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T01:42:54.900 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:54.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:42:54.918 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:54.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:42:54.935 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:54.935 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:42:54.953 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T01:42:54.954 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:54.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:42:54.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:42:54.982 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:54.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:42:54.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:42:55.013 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:55.013 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:42:55.030 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:42:55.087 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T01:42:55.087 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:55.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:42:55.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:42:55.114 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:55.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:42:55.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:42:55.186 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:55.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:42:55.202 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:42:55.215 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T01:42:55.216 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:55.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:42:55.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:42:55.291 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:55.291 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:42:55.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:42:55.364 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:55.365 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:42:55.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:42:55.439 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T01:42:55.520 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T01:42:55.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-09-15T01:42:55.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-09-15T01:42:55.525 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2024-09-15T01:42:55.540 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T01:42:55.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:42:55.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:42:55.545 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:42:55.555 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T01:42:55.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:42:55.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:42:55.600 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:42:55.626 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T01:42:55.626 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T01:42:55.626 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7905049', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'], ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'], ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']], 'seed': 2126, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_00:54:39', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T01:42:55.627 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:42:55.629 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:42:55.631 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:42:55.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:42:55.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:42:55.678 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:42:55.695 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:55.695 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:42:55.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:42:55.702 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-09-15T01:42:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:42:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:55.709 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-15T01:42:55.709 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:55.709 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-15 01:39:17.413787643 +0000 2024-09-15T01:42:55.710 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-15 01:39:17.193792707 +0000 2024-09-15T01:42:55.710 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-15 01:39:17.193792707 +0000 2024-09-15T01:42:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-15T01:42:55.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:42:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-15T01:42:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-15T01:42:55.723 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000427531 s, 1.2 MB/s 2024-09-15T01:42:55.724 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:42:55.734 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-09-15T01:42:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:42:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-15 01:39:17.829778067 +0000 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-15 01:39:17.605783223 +0000 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-15 01:39:17.605783223 +0000 2024-09-15T01:42:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-15T01:42:55.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:42:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-15T01:42:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-15T01:42:55.754 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000419378 s, 1.2 MB/s 2024-09-15T01:42:55.755 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:42:55.812 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-09-15T01:42:55.819 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:42:55.819 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-15T01:42:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-15 01:39:18.277767755 +0000 2024-09-15T01:42:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-15 01:39:18.065772634 +0000 2024-09-15T01:42:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-15 01:39:18.065772634 +0000 2024-09-15T01:42:55.820 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-15T01:42:55.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:42:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-15T01:42:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-15T01:42:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000410631 s, 1.2 MB/s 2024-09-15T01:42:55.879 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:42:55.887 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-15 01:39:18.733757258 +0000 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-15 01:39:18.509762415 +0000 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-15 01:39:18.509762415 +0000 2024-09-15T01:42:55.892 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-15T01:42:55.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:42:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-15T01:42:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-15T01:42:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000370442 s, 1.4 MB/s 2024-09-15T01:42:55.951 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:42:55.960 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:55.960 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:55.960 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:42:55.967 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:42:55.967 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-15T01:42:55.973 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 01:39:20.367081023 +0000 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 01:39:20.167085637 +0000 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 01:39:20.167085637 +0000 2024-09-15T01:42:55.974 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-09-15T01:42:55.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:42:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T01:42:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T01:42:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000305741 s, 1.7 MB/s 2024-09-15T01:42:56.032 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:42:56.041 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-15T01:42:56.046 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 01:39:20.823070505 +0000 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 01:39:20.615075303 +0000 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 01:39:20.615075303 +0000 2024-09-15T01:42:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-09-15T01:42:56.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:42:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T01:42:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T01:42:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000294607 s, 1.7 MB/s 2024-09-15T01:42:56.061 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:42:56.113 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 01:39:21.215061463 +0000 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 01:39:21.003066353 +0000 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 01:39:21.003066353 +0000 2024-09-15T01:42:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-09-15T01:42:56.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:42:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T01:42:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T01:42:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000274117 s, 1.9 MB/s 2024-09-15T01:42:56.136 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:42:56.144 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-15T01:42:56.195 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:42:56.195 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.195 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-15T01:42:56.195 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.196 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 01:39:21.655051314 +0000 2024-09-15T01:42:56.196 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 01:39:21.439056296 +0000 2024-09-15T01:42:56.196 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 01:39:21.439056296 +0000 2024-09-15T01:42:56.196 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-09-15T01:42:56.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:42:56.210 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T01:42:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T01:42:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00029714 s, 1.7 MB/s 2024-09-15T01:42:56.211 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:42:56.218 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:56.218 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:56.219 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:42:56.225 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:42:56.225 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-15 01:39:19.781907941 +0000 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-15 01:39:19.569912828 +0000 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-15 01:39:19.569912828 +0000 2024-09-15T01:42:56.233 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-09-15T01:42:56.234 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:42:56.290 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-15T01:42:56.290 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-15T01:42:56.290 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000423917 s, 1.2 MB/s 2024-09-15T01:42:56.292 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:42:56.299 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-09-15T01:42:56.305 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:42:56.305 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-15 01:39:20.229897615 +0000 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-15 01:39:20.017902501 +0000 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-15 01:39:20.017902501 +0000 2024-09-15T01:42:56.306 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-09-15T01:42:56.306 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:42:56.362 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-15T01:42:56.362 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-15T01:42:56.363 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000401364 s, 1.3 MB/s 2024-09-15T01:42:56.364 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:42:56.371 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 381 Links: 1 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-15 01:39:20.697886827 +0000 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-15 01:39:20.509891160 +0000 2024-09-15T01:42:56.377 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-15 01:39:20.509891160 +0000 2024-09-15T01:42:56.378 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-09-15T01:42:56.378 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:42:56.434 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-15T01:42:56.434 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-15T01:42:56.434 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000393744 s, 1.3 MB/s 2024-09-15T01:42:56.435 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:42:56.443 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 389 Links: 1 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-09-15 01:39:21.137876685 +0000 2024-09-15T01:42:56.449 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-09-15 01:39:20.929881479 +0000 2024-09-15T01:42:56.450 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-09-15 01:39:20.929881479 +0000 2024-09-15T01:42:56.450 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-09-15T01:42:56.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:42:56.506 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-09-15T01:42:56.506 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-09-15T01:42:56.507 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000363242 s, 1.4 MB/s 2024-09-15T01:42:56.507 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:42:56.515 INFO:tasks.ceph:osd dev map: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:56.515 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-15T01:42:56.515 INFO:tasks.ceph:Generating config... 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T01:42:56.516 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd max backfills = 9 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T01:42:56.517 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-15T01:42:56.518 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-15T01:42:56.518 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T01:42:56.518 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T01:42:56.518 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-15T01:42:56.518 INFO:tasks.ceph:[global] ms tcp read timeout = 5 2024-09-15T01:42:56.518 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-09-15T01:42:56.518 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T01:42:56.518 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T01:42:56.518 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T01:42:56.519 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T01:42:56.519 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T01:42:56.519 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T01:42:56.519 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T01:42:56.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T01:42:56.555 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T01:42:56.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T01:42:56.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:42:56.653 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.47'), ('mon.c', '172.21.15.155')] 2024-09-15T01:42:56.653 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.47,172.21.15.155', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-15T01:42:56.654 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:56.654 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:42:56.661 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.47 --add c 172.21.15.155 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:56.741 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:56.741 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = octopus 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:fsid d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2024-09-15T01:42:56.739871+0000 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-09-15T01:42:56.739871+0000 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 15 (octopus) 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.c 2024-09-15T01:42:56.742 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T01:42:56.743 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:42:56.748 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d386169c-ec95-4076-a6ad-eaab3ccf750b... 2024-09-15T01:42:56.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:42:56.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:42:56.754 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:42:56.783 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-09-15T01:42:56.783 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-09-15T01:42:56.783 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.784 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:42:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-09-15T01:42:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.47,172.21.15.155" 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: mon client directed command retry = 5 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: mon election default strategy = 1 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject socket failures = 2500 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: ms tcp read timeout = 5 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-09-15T01:42:56.787 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 120 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-09-15T01:42:56.788 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-09-15T01:42:56.789 INFO:teuthology.orchestra.run.smithi033.stdout: debug monc = 20 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bdev async discard = True 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bdev enable discard = True 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = bitmap 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore zero block detection = True 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 20 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 20 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 10 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T01:42:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify cached snaps = True 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify missing on start = True 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd max backfills = 9 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd max markdown count = 1000 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock override recovery settings = True 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd op queue = debug_random 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd op queue cut off = debug_random 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub min interval = 60 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-09-15T01:42:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-09-15T01:42:56.792 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: mon scrub interval = 300 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-09-15T01:42:56.794 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-09-15T01:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-09-15T01:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-09-15T01:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-09-15T01:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-09-15T01:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.33,172.21.15.47,172.21.15.155" 2024-09-15T01:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: mon client directed command retry = 5 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: mon election default strategy = 1 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: ms inject socket failures = 2500 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: ms tcp read timeout = 5 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 120 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-09-15T01:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: debug monc = 20 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-09-15T01:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore zero block detection = True 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify cached snaps = True 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify missing on start = True 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: osd max backfills = 9 2024-09-15T01:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: osd max markdown count = 1000 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock override recovery settings = True 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: osd op queue = debug_random 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: osd op queue cut off = debug_random 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub min interval = 60 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-09-15T01:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: mon scrub interval = 300 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T01:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-09-15T01:42:56.806 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-09-15T01:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-09-15T01:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.33,172.21.15.47,172.21.15.155" 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: mon client directed command retry = 5 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: mon election default strategy = 1 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: ms inject socket failures = 2500 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: ms tcp read timeout = 5 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: osd_async_recovery_min_cost = 1 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-09-15T01:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 120 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-09-15T01:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-09-15T01:42:56.812 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug monc = 20 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore zero block detection = True 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 20 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 20 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-09-15T01:42:56.813 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 10 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify cached snaps = True 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify missing on start = True 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd max backfills = 9 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd max markdown count = 1000 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock override recovery settings = True 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd op queue = debug_random 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd op queue cut off = debug_random 2024-09-15T01:42:56.814 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub min interval = 60 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-09-15T01:42:56.815 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.816 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: mon scrub interval = 300 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-09-15T01:42:56.817 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:42:56.818 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:42:56.818 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-09-15T01:42:56.818 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-09-15T01:42:56.818 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-09-15T01:42:56.820 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T01:42:56.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T01:42:56.911 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T01:42:56.911 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:56.911 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T01:42:56.919 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:56.919 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T01:42:56.925 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:42:56.926 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:56.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:42:56.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:42:56.949 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:42:56.949 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:56.956 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:42:56.957 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:56.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:42:56.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:42:56.985 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:42:56.985 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:57.038 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:42:57.039 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:57.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:42:57.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:42:57.064 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:42:57.064 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:57.069 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T01:42:57.069 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T01:42:57.069 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:42:57.116 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:42:57.119 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T01:42:57.119 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T01:42:57.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T01:42:57.154 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T01:42:57.168 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T01:42:57.168 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-15T01:42:57.168 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T01:42:57.228 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:57.228 INFO:tasks.ceph:role: osd.0 2024-09-15T01:42:57.228 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:42:57.229 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:42:57.748 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:42:57.748 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:42:57.749 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:42:57.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-15T01:42:57.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T01:42:57.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T01:42:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:42:57.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:42:57.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T01:42:57.910 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:57.910 INFO:tasks.ceph:role: osd.1 2024-09-15T01:42:57.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:42:57.910 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:42:58.428 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:42:58.428 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:42:58.428 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:42:58.429 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:42:58.430 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-15T01:42:58.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T01:42:58.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T01:42:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:42:58.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:42:58.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T01:42:58.533 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:58.533 INFO:tasks.ceph:role: osd.2 2024-09-15T01:42:58.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:42:58.534 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:42:58.996 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:42:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:42:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:42:58.997 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:42:58.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-15T01:42:58.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T01:42:59.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T01:42:59.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:42:59.058 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:42:59.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T01:42:59.116 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T01:42:59.116 INFO:tasks.ceph:role: osd.3 2024-09-15T01:42:59.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:42:59.116 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:42:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:42:59.600 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:42:59.600 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:42:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:42:59.600 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:42:59.601 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-15T01:42:59.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T01:42:59.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T01:42:59.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:42:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:42:59.677 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:42:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.794+0000 7f8b86e29540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T01:42:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.798+0000 7f8b86e29540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T01:42:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.798+0000 7f8b86e29540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T01:42:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.798+0000 7f8b86e29540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T01:42:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.798+0000 7f8b86e29540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T01:42:59.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:42:59.802+0000 7f8b86e29540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T01:43:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:00.374+0000 7f8b86e29540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:01.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:01.149+0000 7f8b86e29540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:01.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:01.929+0000 7f8b86e29540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:02.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T01:43:02.799 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T01:43:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T01:43:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T01:43:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T01:43:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T01:43:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:02.837+0000 7fa4707e9540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T01:43:03.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:03.401+0000 7fa4707e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:04.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:04.189+0000 7fa4707e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:04.969+0000 7fa4707e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:05.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T01:43:05.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T01:43:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:05.921+0000 7fc75949d540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T01:43:06.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:06.485+0000 7fc75949d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:07.277+0000 7fc75949d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:08.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:08.057+0000 7fc75949d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:08.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T01:43:08.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.001+0000 7fd21630c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T01:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.005+0000 7fd21630c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T01:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.005+0000 7fd21630c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T01:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.005+0000 7fd21630c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T01:43:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.005+0000 7fd21630c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T01:43:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.005+0000 7fd21630c540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T01:43:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:09.565+0000 7fd21630c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:10.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:10.349+0000 7fd21630c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:11.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:43:11.133+0000 7fd21630c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:11.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T01:43:11.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T01:43:12.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:12.012 INFO:tasks.ceph:role: osd.4 2024-09-15T01:43:12.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:43:12.013 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:43:12.100 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:12.101 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:12.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T01:43:12.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T01:43:12.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T01:43:12.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:12.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T01:43:12.144 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:12.144 INFO:tasks.ceph:role: osd.5 2024-09-15T01:43:12.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:43:12.144 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:43:12.183 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:12.183 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:12.183 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:12.184 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:12.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T01:43:12.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T01:43:12.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T01:43:12.260 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:12.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:12.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T01:43:12.273 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:12.273 INFO:tasks.ceph:role: osd.6 2024-09-15T01:43:12.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:43:12.274 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:43:12.314 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:12.314 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:12.315 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:12.316 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T01:43:12.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T01:43:12.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T01:43:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:12.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:12.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T01:43:12.404 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:12.404 INFO:tasks.ceph:role: osd.7 2024-09-15T01:43:12.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:43:12.404 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:12.446 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:12.446 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:12.446 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:12.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T01:43:12.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T01:43:12.465 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T01:43:12.479 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:12.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:12.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.602+0000 7fb99f910540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T01:43:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.602+0000 7fb99f910540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T01:43:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.602+0000 7fb99f910540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T01:43:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.602+0000 7fb99f910540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T01:43:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.602+0000 7fb99f910540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T01:43:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:12.606+0000 7fb99f910540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T01:43:13.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:13.158+0000 7fb99f910540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:13.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:13.937+0000 7fb99f910540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:14.721+0000 7fb99f910540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:15.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T01:43:15.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T01:43:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T01:43:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T01:43:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T01:43:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T01:43:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:15.613+0000 7f1be71b6540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T01:43:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:16.173+0000 7f1be71b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:16.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:16.957+0000 7f1be71b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:17.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:17.745+0000 7f1be71b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:18.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T01:43:18.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:18.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T01:43:18.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T01:43:18.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T01:43:18.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T01:43:18.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T01:43:18.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:18.689+0000 7f3d3b00f540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T01:43:19.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:19.249+0000 7f3d3b00f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:20.057+0000 7f3d3b00f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:20.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:20.837+0000 7f3d3b00f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:21.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T01:43:21.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:21.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.785+0000 7f4505973540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T01:43:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.789+0000 7f4505973540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T01:43:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.789+0000 7f4505973540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T01:43:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.789+0000 7f4505973540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T01:43:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.789+0000 7f4505973540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T01:43:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:21.789+0000 7f4505973540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T01:43:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:22.353+0000 7f4505973540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:23.133+0000 7f4505973540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:23.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T01:43:23.917+0000 7f4505973540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:24.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T01:43:24.788 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-15T01:43:24.805 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:24.805 INFO:tasks.ceph:role: osd.8 2024-09-15T01:43:24.806 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:43:24.806 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:43:25.387 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:25.387 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:25.387 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:25.387 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 2024-09-15T01:43:25.387 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:25.388 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:25.388 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:25.388 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:25.388 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:25.388 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:25.389 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-09-15T01:43:25.389 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-8 2024-09-15T01:43:25.503 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-15T01:43:25.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:25.516 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:25.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-15T01:43:25.526 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:25.526 INFO:tasks.ceph:role: osd.9 2024-09-15T01:43:25.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:43:25.526 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:26.058 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:26.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-09-15T01:43:26.060 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-9 2024-09-15T01:43:26.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-15T01:43:26.152 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:26.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:26.153 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-15T01:43:26.163 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:26.163 INFO:tasks.ceph:role: osd.10 2024-09-15T01:43:26.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:43:26.163 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:26.618 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:26.619 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:26.619 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:26.619 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:26.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-09-15T01:43:26.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-15T01:43:26.687 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-15T01:43:26.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:26.699 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:26.699 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-15T01:43:26.711 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T01:43:26.711 INFO:tasks.ceph:role: osd.11 2024-09-15T01:43:26.711 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:43:26.711 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T01:43:27.203 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:43:27.203 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:43:27.203 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:43:27.203 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 2024-09-15T01:43:27.203 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:43:27.204 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:43:27.204 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:43:27.204 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T01:43:27.204 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:43:27.204 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:43:27.205 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-09-15T01:43:27.205 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-11 2024-09-15T01:43:27.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-15T01:43:27.280 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T01:43:27.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:43:27.281 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:27.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.352+0000 7fe90c127540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-15T01:43:27.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.352+0000 7fe90c127540 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-15T01:43:27.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.352+0000 7fe90c127540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-15T01:43:27.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.352+0000 7fe90c127540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-15T01:43:27.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.352+0000 7fe90c127540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-15T01:43:27.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.360+0000 7fe90c127540 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-15T01:43:27.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:27.924+0000 7fe90c127540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:28.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:28.720+0000 7fe90c127540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:29.506 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:29.500+0000 7fe90c127540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:30.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-15T01:43:30.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:30.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.404+0000 7f3beddd1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-15T01:43:30.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.404+0000 7f3beddd1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-15T01:43:30.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.404+0000 7f3beddd1540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-15T01:43:30.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.404+0000 7f3beddd1540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-15T01:43:30.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.404+0000 7f3beddd1540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-15T01:43:30.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.408+0000 7f3beddd1540 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-15T01:43:30.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:30.976+0000 7f3beddd1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:31.774 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:31.768+0000 7f3beddd1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:32.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:32.552+0000 7f3beddd1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:33.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-15T01:43:33.416 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:33.499 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.492+0000 7f6a3918f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-15T01:43:33.499 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.492+0000 7f6a3918f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-15T01:43:33.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.492+0000 7f6a3918f540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-15T01:43:33.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.492+0000 7f6a3918f540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-15T01:43:33.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.492+0000 7f6a3918f540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-15T01:43:33.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:33.496+0000 7f6a3918f540 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-15T01:43:34.063 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:34.056+0000 7f6a3918f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:34.862 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:34.856+0000 7f6a3918f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:35.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:35.660+0000 7f6a3918f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:36.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-15T01:43:36.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:36.569 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-15T01:43:36.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-15T01:43:36.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-15T01:43:36.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-15T01:43:36.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-15T01:43:36.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:36.564+0000 7fe09c0f6540 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-15T01:43:37.124 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:37.116+0000 7fe09c0f6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:37.909 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:37.904+0000 7fe09c0f6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:38.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-09-15T01:43:38.680+0000 7fe09c0f6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T01:43:39.527 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-15T01:43:39.588 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T01:43:39.588 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:39.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T01:43:39.605 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:39.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T01:43:39.664 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:39.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T01:43:39.674 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:39.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T01:43:39.688 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:39.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T01:43:39.705 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:39.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T01:43:39.763 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:39.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T01:43:39.773 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:39.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T01:43:39.789 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:39.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T01:43:39.800 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:39.800 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-15T01:43:39.817 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:39.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-15T01:43:39.831 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:39.832 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-15T01:43:39.846 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:39.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-15T01:43:39.906 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:39.906 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T01:43:39.913 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T01:43:39.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:43:39.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:43:39.960 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCjO+Zm3w9wLxAAFDI8Z3N6Jcz9KOXZ1hxVCQ== 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCmO+ZmLt4YMhAAxDUfWugBgRPGeAIB/TWV/A== 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCpO+ZmULUiNxAArh0udhvLJSVKra9O5e6oJA== 2024-09-15T01:43:39.970 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCtO+ZmCyZdABAAM22tb42RUVe+dD235Izw0A== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQChO+ZmHtrYBhAAB5eG/xCYfw3tOG58qSusSw== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCwO+ZmT+wwJBAAdCELehFRBajUBjDEn+YAmA== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCzO+ZmeTjSJBAA4lBHJFKez8kgBD+k3SvGsg== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC2O+ZmyhZGKRAAZ5m2vPGR9lavghl8J1o3Yw== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC5O+Zm7dwsLxAA13A2IAi0SGgYeoDmhw6TYw== 2024-09-15T01:43:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.8] 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC/O+ZmBn9cFRAAgnLPfBz+vMwlbum9lcCdzg== 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.9] 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDCO+Zml/BqGBAAz7U3Jvni847XXZ+7QV5FHg== 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.10] 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDFO+Zmm6yrHRAAQXqYDry+rdskElaETby1pw== 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.11] 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDIO+Zm4cfeIRAAbWlkAgO6omRFS6AHBibzmg== 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-09-15T01:43:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQChO+ZmLz4YCRAAgJ+5mdHg7lTM+Jw9w2YYUQ== 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCjO+Zm3w9wLxAAFDI8Z3N6Jcz9KOXZ1hxVCQ== 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCmO+ZmLt4YMhAAxDUfWugBgRPGeAIB/TWV/A== 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCpO+ZmULUiNxAArh0udhvLJSVKra9O5e6oJA== 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCtO+ZmCyZdABAAM22tb42RUVe+dD235Izw0A== 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-09-15T01:43:39.973 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQChO+ZmHtrYBhAAB5eG/xCYfw3tOG58qSusSw== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCwO+ZmT+wwJBAAdCELehFRBajUBjDEn+YAmA== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCzO+ZmeTjSJBAA4lBHJFKez8kgBD+k3SvGsg== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC2O+ZmyhZGKRAAZ5m2vPGR9lavghl8J1o3Yw== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC5O+Zm7dwsLxAA13A2IAi0SGgYeoDmhw6TYw== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.8] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC/O+ZmBn9cFRAAgnLPfBz+vMwlbum9lcCdzg== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.9] 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDCO+Zml/BqGBAAz7U3Jvni847XXZ+7QV5FHg== 2024-09-15T01:43:39.974 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.10] 2024-09-15T01:43:39.975 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDFO+Zmm6yrHRAAQXqYDry+rdskElaETby1pw== 2024-09-15T01:43:39.975 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.11] 2024-09-15T01:43:39.975 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDIO+Zm4cfeIRAAbWlkAgO6omRFS6AHBibzmg== 2024-09-15T01:43:39.975 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-09-15T01:43:39.975 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQChO+ZmLz4YCRAAgJ+5mdHg7lTM+Jw9w2YYUQ== 2024-09-15T01:43:39.977 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCjO+Zm3w9wLxAAFDI8Z3N6Jcz9KOXZ1hxVCQ== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCmO+ZmLt4YMhAAxDUfWugBgRPGeAIB/TWV/A== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCpO+ZmULUiNxAArh0udhvLJSVKra9O5e6oJA== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCtO+ZmCyZdABAAM22tb42RUVe+dD235Izw0A== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQChO+ZmHtrYBhAAB5eG/xCYfw3tOG58qSusSw== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCwO+ZmT+wwJBAAdCELehFRBajUBjDEn+YAmA== 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2024-09-15T01:43:39.978 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCzO+ZmeTjSJBAA4lBHJFKez8kgBD+k3SvGsg== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQC2O+ZmyhZGKRAAZ5m2vPGR9lavghl8J1o3Yw== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQC5O+Zm7dwsLxAA13A2IAi0SGgYeoDmhw6TYw== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.8] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQC/O+ZmBn9cFRAAgnLPfBz+vMwlbum9lcCdzg== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.9] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDCO+Zml/BqGBAAz7U3Jvni847XXZ+7QV5FHg== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.10] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDFO+Zmm6yrHRAAQXqYDry+rdskElaETby1pw== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.11] 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDIO+Zm4cfeIRAAbWlkAgO6omRFS6AHBibzmg== 2024-09-15T01:43:39.979 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2024-09-15T01:43:39.980 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQChO+ZmLz4YCRAAgJ+5mdHg7lTM+Jw9w2YYUQ== 2024-09-15T01:43:39.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:39.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.022 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.161 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:43:40.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:43:40.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:43:40.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.470 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.698 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.832 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.918 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:43:40.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:43:40.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:43:40.968 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:43:41.056 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T01:43:41.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T01:43:41.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:43:41.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T01:43:41.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T01:43:41.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:43:41.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T01:43:41.265 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T01:43:41.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:43:41.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T01:43:41.351 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:41.354 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:41.356 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:43:41.398 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T01:43:41.398 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:43:41.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:43:41.401 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:43:41.401 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T01:43:41.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T01:43:41.406 INFO:tasks.ceph.mon.b:Started 2024-09-15T01:43:41.407 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T01:43:41.407 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T01:43:41.409 INFO:tasks.ceph.mon.c:Started 2024-09-15T01:43:41.409 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T01:43:41.409 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T01:43:41.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T01:43:41.411 INFO:tasks.ceph.mgr.x:Started 2024-09-15T01:43:41.412 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T01:43:41.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T01:43:41.857 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-09-15T01:43:41.871 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T01:43:41.871 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:41.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T01:43:41.883 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:41.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T01:43:41.937 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:41.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T01:43:41.953 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-15T01:43:41.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T01:43:41.966 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:41.961+0000 7f065df61000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T01:43:41.969 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:41.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T01:43:41.985 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:41.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T01:43:42.043 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:42.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T01:43:42.055 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T01:43:42.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T01:43:42.064 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:42.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-15T01:43:42.080 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:42.080 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-15T01:43:42.143 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:42.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-15T01:43:42.155 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-09-15T01:43:42.155 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-15T01:43:42.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 5bbf5b49-bace-424f-b96c-91c3a25e48e0 0 2024-09-15T01:43:42.384 INFO:teuthology.orchestra.run.smithi155.stdout:0 2024-09-15T01:43:42.398 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464 1 2024-09-15T01:43:42.615 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-09-15T01:43:42.630 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new fcc939b7-c3fb-4117-af41-9eec226f3020 2 2024-09-15T01:43:42.753 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:42.749+0000 7f065df61000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T01:43:42.812 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:42.809+0000 7f065df61000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T01:43:42.837 INFO:teuthology.orchestra.run.smithi155.stdout:2 2024-09-15T01:43:42.851 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 57f1d668-f85b-4b0c-9093-fb5fef4b91ef 3 2024-09-15T01:43:42.931 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:42.925+0000 7f065df61000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:43:43.053 INFO:teuthology.orchestra.run.smithi155.stdout:3 2024-09-15T01:43:43.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new a274d12c-8683-491d-8656-5af5d12164ee 4 2024-09-15T01:43:43.176 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.169+0000 7f065df61000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T01:43:43.230 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.225+0000 7f065df61000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T01:43:43.270 INFO:teuthology.orchestra.run.smithi155.stdout:4 2024-09-15T01:43:43.284 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new f0a90a47-9b16-46f6-a2a7-afa4a6f2a298 5 2024-09-15T01:43:43.430 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.425+0000 7f065df61000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T01:43:43.492 INFO:teuthology.orchestra.run.smithi155.stdout:5 2024-09-15T01:43:43.498 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.493+0000 7f065df61000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T01:43:43.506 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 194e23d1-7f52-43ee-8557-52d75a3c22dc 6 2024-09-15T01:43:43.566 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.561+0000 7f065df61000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T01:43:43.717 INFO:teuthology.orchestra.run.smithi155.stdout:6 2024-09-15T01:43:43.731 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 1d1c61e7-be03-4c72-bc95-e187eb95a1b8 7 2024-09-15T01:43:43.932 INFO:teuthology.orchestra.run.smithi155.stdout:7 2024-09-15T01:43:43.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 1f23463d-2214-416b-b3ba-23132f2d84ee 8 2024-09-15T01:43:43.996 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:43.989+0000 7f065df61000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T01:43:44.121 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.117+0000 7f065df61000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T01:43:44.150 INFO:teuthology.orchestra.run.smithi155.stdout:8 2024-09-15T01:43:44.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 39226fc7-a654-41c6-9d70-5ea2f69c236b 9 2024-09-15T01:43:44.373 INFO:teuthology.orchestra.run.smithi155.stdout:9 2024-09-15T01:43:44.387 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new eaafc17e-407d-43aa-8083-2c56660d7db4 10 2024-09-15T01:43:44.451 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.445+0000 7f065df61000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T01:43:44.592 INFO:teuthology.orchestra.run.smithi155.stdout:10 2024-09-15T01:43:44.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 05fdd01d-8c78-4a61-81e7-1d9816e951a3 11 2024-09-15T01:43:44.626 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.621+0000 7f065df61000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:43:44.695 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.689+0000 7f065df61000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T01:43:44.749 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.745+0000 7f065df61000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T01:43:44.809 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.805+0000 7f065df61000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T01:43:44.810 INFO:teuthology.orchestra.run.smithi155.stdout:11 2024-09-15T01:43:44.824 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T01:43:44.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T01:43:44.826 INFO:tasks.ceph.osd.0:Started 2024-09-15T01:43:44.826 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T01:43:44.826 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T01:43:44.829 INFO:tasks.ceph.osd.1:Started 2024-09-15T01:43:44.829 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T01:43:44.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T01:43:44.832 INFO:tasks.ceph.osd.2:Started 2024-09-15T01:43:44.832 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T01:43:44.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T01:43:44.834 INFO:tasks.ceph.osd.3:Started 2024-09-15T01:43:44.835 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T01:43:44.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T01:43:44.837 INFO:tasks.ceph.osd.4:Started 2024-09-15T01:43:44.837 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T01:43:44.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T01:43:44.839 INFO:tasks.ceph.osd.5:Started 2024-09-15T01:43:44.840 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T01:43:44.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T01:43:44.841 INFO:tasks.ceph.osd.6:Started 2024-09-15T01:43:44.842 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T01:43:44.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T01:43:44.843 INFO:tasks.ceph.osd.7:Started 2024-09-15T01:43:44.844 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-15T01:43:44.844 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-15T01:43:44.846 INFO:tasks.ceph.osd.8:Started 2024-09-15T01:43:44.846 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-15T01:43:44.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-15T01:43:44.848 INFO:tasks.ceph.osd.9:Started 2024-09-15T01:43:44.848 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-15T01:43:44.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-15T01:43:44.851 INFO:tasks.ceph.osd.10:Started 2024-09-15T01:43:44.851 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-15T01:43:44.851 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-15T01:43:44.853 INFO:tasks.ceph.osd.11:Started 2024-09-15T01:43:44.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:43:44.974 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:44.969+0000 7f065df61000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T01:43:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:43:45.045 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":14,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:43:44.800329+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:43:44.800329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:43:45.057 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T01:43:45.057 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T01:43:45.152 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:45.145+0000 7f065df61000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T01:43:45.194 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:43:45.189+0000 7fc4d0ce4540 -1 Falling back to public interface 2024-09-15T01:43:45.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:43:45.208+0000 7fa17b887540 -1 Falling back to public interface 2024-09-15T01:43:45.215 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:43:45.208+0000 7f84d0113540 -1 Falling back to public interface 2024-09-15T01:43:45.215 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:43:45.208+0000 7f69e865d540 -1 Falling back to public interface 2024-09-15T01:43:45.226 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:45.221+0000 7f065df61000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T01:43:45.227 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:43:45.221+0000 7fdbfee03540 -1 Falling back to public interface 2024-09-15T01:43:45.404 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:45.397+0000 7f065df61000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T01:43:45.446 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:43:45.440+0000 7f2b18a18540 -1 Falling back to public interface 2024-09-15T01:43:45.447 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:43:45.440+0000 7fb3d8ac6540 -1 Falling back to public interface 2024-09-15T01:43:45.447 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:43:45.441+0000 7f4fef5f9540 -1 Falling back to public interface 2024-09-15T01:43:45.458 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:43:45.452+0000 7f17cdbc7540 -1 Falling back to public interface 2024-09-15T01:43:45.462 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:43:45.456+0000 7f1c1dab7540 -1 Falling back to public interface 2024-09-15T01:43:45.470 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:43:45.465+0000 7f785de56540 -1 Falling back to public interface 2024-09-15T01:43:45.480 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:45.477+0000 7f065df61000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T01:43:45.486 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:43:45.480+0000 7fe4e9b84540 -1 Falling back to public interface 2024-09-15T01:43:46.248 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:46.241+0000 7f065df61000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T01:43:46.267 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:43:46.451 INFO:teuthology.misc.health.smithi033.stdout: 2024-09-15T01:43:46.452 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":14,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:43:44.800329+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:43:44.800329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:43:46.463 DEBUG:teuthology.misc:0 of 12 OSDs are up 2024-09-15T01:43:47.289 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:47.285+0000 7f0646fd1700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T01:43:47.289 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T01:43:47.285+0000 7f0646fd1700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T01:43:48.695 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:43:48.688+0000 7f84d0113540 -1 osd.8 0 log_to_monitors true 2024-09-15T01:43:48.979 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:43:48.973+0000 7fc4d0ce4540 -1 osd.4 0 log_to_monitors true 2024-09-15T01:43:49.011 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:43:49.004+0000 7f69e865d540 -1 osd.9 0 log_to_monitors true 2024-09-15T01:43:49.030 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:43:49.024+0000 7f17cdbc7540 -1 osd.2 0 log_to_monitors true 2024-09-15T01:43:49.048 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:43:49.041+0000 7fdbfee03540 -1 osd.6 0 log_to_monitors true 2024-09-15T01:43:49.054 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:43:49.048+0000 7fa17b887540 -1 osd.0 0 log_to_monitors true 2024-09-15T01:43:49.167 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:43:49.161+0000 7f4fef5f9540 -1 osd.5 0 log_to_monitors true 2024-09-15T01:43:49.189 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:43:49.184+0000 7f2b18a18540 -1 osd.10 0 log_to_monitors true 2024-09-15T01:43:49.191 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:43:49.184+0000 7fb3d8ac6540 -1 osd.1 0 log_to_monitors true 2024-09-15T01:43:49.214 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:43:49.208+0000 7f1c1dab7540 -1 osd.11 0 log_to_monitors true 2024-09-15T01:43:49.259 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:43:49.252+0000 7fe4e9b84540 -1 osd.3 0 log_to_monitors true 2024-09-15T01:43:49.276 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:43:49.269+0000 7f785de56540 -1 osd.7 0 log_to_monitors true 2024-09-15T01:43:50.309 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:43:50.304+0000 7f84cb8b3700 -1 osd.8 0 waiting for initial osdmap 2024-09-15T01:43:50.309 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:43:50.304+0000 7f69e3dfd700 -1 osd.9 0 waiting for initial osdmap 2024-09-15T01:43:50.310 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:43:50.304+0000 7f2b141b8700 -1 osd.10 0 waiting for initial osdmap 2024-09-15T01:43:50.310 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:43:50.304+0000 7f1c19257700 -1 osd.11 0 waiting for initial osdmap 2024-09-15T01:43:50.311 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:43:50.305+0000 7f4fead99700 -1 osd.5 0 waiting for initial osdmap 2024-09-15T01:43:50.311 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:43:50.304+0000 7fa17884c700 -1 osd.0 0 waiting for initial osdmap 2024-09-15T01:43:50.311 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:43:50.305+0000 7fdbfa5a3700 -1 osd.6 0 waiting for initial osdmap 2024-09-15T01:43:50.311 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:43:50.304+0000 7fb3d4266700 -1 osd.1 0 waiting for initial osdmap 2024-09-15T01:43:50.312 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:43:50.305+0000 7fc4cc484700 -1 osd.4 0 waiting for initial osdmap 2024-09-15T01:43:50.312 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:43:50.304+0000 7f17cab8c700 -1 osd.2 0 waiting for initial osdmap 2024-09-15T01:43:50.317 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:43:50.312+0000 7fa1705cb700 -1 osd.0 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.317 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:43:50.308+0000 7fe4e6b49700 -1 osd.3 0 waiting for initial osdmap 2024-09-15T01:43:50.317 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:43:50.312+0000 7fb3cd009700 -1 osd.1 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.318 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:43:50.312+0000 7f84c5658700 -1 osd.8 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.318 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:43:50.312+0000 7f2b0cf5b700 -1 osd.10 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.318 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:43:50.312+0000 7f69dd3a1700 -1 osd.9 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.319 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:43:50.312+0000 7f1c11ffa700 -1 osd.11 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.319 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:43:50.313+0000 7f4fe3b3c700 -1 osd.5 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.319 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:43:50.312+0000 7f17c290b700 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.320 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:43:50.313+0000 7fdbf3b47700 -1 osd.6 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.320 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:43:50.313+0000 7fc4c5a28700 -1 osd.4 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:50.323 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:43:50.316+0000 7fe4de0c7700 -1 osd.3 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:51.316 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:43:51.309+0000 7f78595f6700 -1 osd.7 0 waiting for initial osdmap 2024-09-15T01:43:51.323 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:43:51.317+0000 7f7852399700 -1 osd.7 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:43:53.675 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:43:53.977 INFO:teuthology.misc.health.smithi033.stdout: 2024-09-15T01:43:53.977 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":19,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:43:53.361820+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:43:44.800329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.101167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.166075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.012320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.227835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.010828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.170791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.037098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.281668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:49.661197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.009199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.143021+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.187329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:43:53.990 DEBUG:teuthology.misc:12 of 12 OSDs are up 2024-09-15T01:43:53.990 INFO:tasks.ceph:Creating RBD pool 2024-09-15T01:43:53.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T01:43:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'rbd' created 2024-09-15T01:43:54.406 DEBUG:teuthology.orchestra.run.smithi033:> rbd --cluster ceph pool init rbd 2024-09-15T01:43:57.424 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T01:43:57.425 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T01:43:57.425 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T01:43:57.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T01:43:57.426 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T01:43:57.765 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:43:57.783 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":21989},{"type":"v1","addr":"172.21.15.47:6833","nonce":21989}]},"active_addr":"172.21.15.47:6833/21989","active_change":"2024-09-15T01:43:46.266263+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1090304122}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":94595151}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2370716689}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2047575467}]}]} 2024-09-15T01:43:57.785 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T01:43:57.785 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T01:43:57.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:43:58.081 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:43:58.081 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:43:57.395347+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:43:44.800329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.101167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.166075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.012320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.227835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.010828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.170791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.037098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.281668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:49.661197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.009199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.143021+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.187329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:43:58.094 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T01:43:58.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:43:58.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:43:58.387 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:43:57.395347+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:43:44.800329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.101167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.166075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.012320+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.227835+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.010828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.170791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.037098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.281668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:49.661197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.009199+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.143021+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:43:50.187329+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:43:58.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T01:43:58.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T01:43:58.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T01:43:58.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T01:43:58.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T01:43:58.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T01:43:58.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T01:43:58.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T01:43:58.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T01:43:58.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T01:43:58.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T01:43:58.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T01:43:58.662 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:43:58.670 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:43:58.671 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:43:58.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:43:58.678 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:43:58.691 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:43:58.706 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:43:58.730 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:43:58.740 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:43:58.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:43:58.741 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:58.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:43:58.748 INFO:teuthology.orchestra.run.smithi033.stdout:77309411333 2024-09-15T01:43:58.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:43:58.750 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:43:58.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:43:58.753 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:43:58.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:43:59.256 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.0 2024-09-15T01:43:59.312 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:59.328 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.6 2024-09-15T01:43:59.342 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.4 2024-09-15T01:43:59.363 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.363 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.11 2024-09-15T01:43:59.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.3 2024-09-15T01:43:59.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.1 2024-09-15T01:43:59.394 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:59.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.2 2024-09-15T01:43:59.407 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.412 INFO:teuthology.orchestra.run.smithi033.stdout:77309411331 2024-09-15T01:43:59.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.8 2024-09-15T01:43:59.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411331 for osd.7 2024-09-15T01:43:59.444 INFO:teuthology.orchestra.run.smithi033.stdout:73014444034 2024-09-15T01:43:59.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444034 for osd.9 2024-09-15T01:43:59.495 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:59.509 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:43:59.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.5 2024-09-15T01:43:59.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.10 2024-09-15T01:44:00.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:44:00.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:44:00.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:44:00.378 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:44:00.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:44:00.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:44:00.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:44:00.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:44:00.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:44:00.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:44:00.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:44:00.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:44:00.809 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:00.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.0 2024-09-15T01:44:00.839 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:00.887 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:44:00.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.6 2024-09-15T01:44:00.921 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:00.948 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:00.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.4 2024-09-15T01:44:00.986 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:00.986 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:00.988 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:01.003 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:01.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.1 2024-09-15T01:44:01.005 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.007 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:44:01.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.11 2024-09-15T01:44:01.021 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.3 2024-09-15T01:44:01.026 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.2 2024-09-15T01:44:01.029 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.081 INFO:teuthology.orchestra.run.smithi033.stdout:77309411333 2024-09-15T01:44:01.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.7 2024-09-15T01:44:01.097 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.110 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:01.120 INFO:teuthology.orchestra.run.smithi033.stdout:73014444035 2024-09-15T01:44:01.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.8 2024-09-15T01:44:01.121 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.123 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:44:01.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.9 2024-09-15T01:44:01.133 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.10 2024-09-15T01:44:01.135 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.162 INFO:teuthology.orchestra.run.smithi033.stdout:73014444036 2024-09-15T01:44:01.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.5 2024-09-15T01:44:01.174 DEBUG:teuthology.parallel:result is None 2024-09-15T01:44:01.174 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:44:01.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:01.454 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:01.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-15T01:44:00.283746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3572344,"kb_used_data":3728,"kb_used_omap":19,"kb_used_meta":3568108,"kb_avail":1128889736,"statfs":{"total":1159641169920,"available":1155983089664,"internally_reserved":0,"allocated":3817472,"data_stored":2126842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3653742624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.893027"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903312+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:43:57.903312+0000","last_peered":"2024-09-15T01:43:57.903312+0000","last_clean":"2024-09-15T01:43:57.903312+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:43:57.903312+0000","last_undegraded":"2024-09-15T01:43:57.903312+0000","last_fullsized":"2024-09-15T01:43:57.903312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416834+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:43:57.416834+0000","last_peered":"2024-09-15T01:43:57.416834+0000","last_clean":"2024-09-15T01:43:57.416834+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:43:57.416834+0000","last_undegraded":"2024-09-15T01:43:57.416834+0000","last_fullsized":"2024-09-15T01:43:57.416834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407762+0000","last_change":"2024-09-15T01:43:57.408162+0000","last_active":"2024-09-15T01:43:57.407762+0000","last_peered":"2024-09-15T01:43:57.407762+0000","last_clean":"2024-09-15T01:43:57.407762+0000","last_became_active":"2024-09-15T01:43:55.500523+0000","last_became_peered":"2024-09-15T01:43:55.500523+0000","last_unstale":"2024-09-15T01:43:57.407762+0000","last_undegraded":"2024-09-15T01:43:57.407762+0000","last_fullsized":"2024-09-15T01:43:57.407762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061773000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416957+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:43:57.416957+0000","last_peered":"2024-09-15T01:43:57.416957+0000","last_clean":"2024-09-15T01:43:57.416957+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:43:57.416957+0000","last_undegraded":"2024-09-15T01:43:57.416957+0000","last_fullsized":"2024-09-15T01:43:57.416957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.415640+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:43:57.415640+0000","last_peered":"2024-09-15T01:43:57.415640+0000","last_clean":"2024-09-15T01:43:57.415640+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:43:57.415640+0000","last_undegraded":"2024-09-15T01:43:57.415640+0000","last_fullsized":"2024-09-15T01:43:57.415640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407639+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:43:57.407639+0000","last_peered":"2024-09-15T01:43:57.407639+0000","last_clean":"2024-09-15T01:43:57.407639+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:43:57.407639+0000","last_undegraded":"2024-09-15T01:43:57.407639+0000","last_fullsized":"2024-09-15T01:43:57.407639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903134+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:43:57.903134+0000","last_peered":"2024-09-15T01:43:57.903134+0000","last_clean":"2024-09-15T01:43:57.903134+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:43:57.903134+0000","last_undegraded":"2024-09-15T01:43:57.903134+0000","last_fullsized":"2024-09-15T01:43:57.903134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.413658+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:43:57.413658+0000","last_peered":"2024-09-15T01:43:57.413658+0000","last_clean":"2024-09-15T01:43:57.413658+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:43:57.413658+0000","last_undegraded":"2024-09-15T01:43:57.413658+0000","last_fullsized":"2024-09-15T01:43:57.413658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":125,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T01:43:56.655181+0000","last_change":"2024-09-15T01:43:54.208642+0000","last_active":"2024-09-15T01:43:56.655181+0000","last_peered":"2024-09-15T01:43:56.655181+0000","last_clean":"2024-09-15T01:43:56.655181+0000","last_became_active":"2024-09-15T01:43:54.206406+0000","last_became_peered":"2024-09-15T01:43:54.206406+0000","last_unstale":"2024-09-15T01:43:56.655181+0000","last_undegraded":"2024-09-15T01:43:56.655181+0000","last_fullsized":"2024-09-15T01:43:56.655181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:58.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":761856,"data_stored":618165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9379999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":757760,"data_stored":617217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":10,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.925}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.871}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.837}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1349999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.925}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]}]},{"osd":5,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":4,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.661}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]}]},{"osd":3,"up_from":17,"seq":73014444035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9330000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2879999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.802}]}]},{"osd":2,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4409999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]}]},{"osd":1,"up_from":17,"seq":73014444035,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]}]},{"osd":0,"up_from":17,"seq":73014444035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":88289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.165}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.929}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:01.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:01.761 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:01.763 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:01.777 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-15T01:44:01.495046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3572344,"kb_used_data":3728,"kb_used_omap":19,"kb_used_meta":3568108,"kb_avail":1128889736,"statfs":{"total":1159641169920,"available":1155983089664,"internally_reserved":0,"allocated":3817472,"data_stored":2126842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3653742624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.104326"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903312+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:43:57.903312+0000","last_peered":"2024-09-15T01:43:57.903312+0000","last_clean":"2024-09-15T01:43:57.903312+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:43:57.903312+0000","last_undegraded":"2024-09-15T01:43:57.903312+0000","last_fullsized":"2024-09-15T01:43:57.903312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416834+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:43:57.416834+0000","last_peered":"2024-09-15T01:43:57.416834+0000","last_clean":"2024-09-15T01:43:57.416834+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:43:57.416834+0000","last_undegraded":"2024-09-15T01:43:57.416834+0000","last_fullsized":"2024-09-15T01:43:57.416834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407762+0000","last_change":"2024-09-15T01:43:57.408162+0000","last_active":"2024-09-15T01:43:57.407762+0000","last_peered":"2024-09-15T01:43:57.407762+0000","last_clean":"2024-09-15T01:43:57.407762+0000","last_became_active":"2024-09-15T01:43:55.500523+0000","last_became_peered":"2024-09-15T01:43:55.500523+0000","last_unstale":"2024-09-15T01:43:57.407762+0000","last_undegraded":"2024-09-15T01:43:57.407762+0000","last_fullsized":"2024-09-15T01:43:57.407762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061773000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416957+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:43:57.416957+0000","last_peered":"2024-09-15T01:43:57.416957+0000","last_clean":"2024-09-15T01:43:57.416957+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:43:57.416957+0000","last_undegraded":"2024-09-15T01:43:57.416957+0000","last_fullsized":"2024-09-15T01:43:57.416957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.415640+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:43:57.415640+0000","last_peered":"2024-09-15T01:43:57.415640+0000","last_clean":"2024-09-15T01:43:57.415640+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:43:57.415640+0000","last_undegraded":"2024-09-15T01:43:57.415640+0000","last_fullsized":"2024-09-15T01:43:57.415640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407639+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:43:57.407639+0000","last_peered":"2024-09-15T01:43:57.407639+0000","last_clean":"2024-09-15T01:43:57.407639+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:43:57.407639+0000","last_undegraded":"2024-09-15T01:43:57.407639+0000","last_fullsized":"2024-09-15T01:43:57.407639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903134+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:43:57.903134+0000","last_peered":"2024-09-15T01:43:57.903134+0000","last_clean":"2024-09-15T01:43:57.903134+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:43:57.903134+0000","last_undegraded":"2024-09-15T01:43:57.903134+0000","last_fullsized":"2024-09-15T01:43:57.903134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.413658+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:43:57.413658+0000","last_peered":"2024-09-15T01:43:57.413658+0000","last_clean":"2024-09-15T01:43:57.413658+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:43:57.413658+0000","last_undegraded":"2024-09-15T01:43:57.413658+0000","last_fullsized":"2024-09-15T01:43:57.413658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":125,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T01:43:56.655181+0000","last_change":"2024-09-15T01:43:54.208642+0000","last_active":"2024-09-15T01:43:56.655181+0000","last_peered":"2024-09-15T01:43:56.655181+0000","last_clean":"2024-09-15T01:43:56.655181+0000","last_became_active":"2024-09-15T01:43:54.206406+0000","last_became_peered":"2024-09-15T01:43:54.206406+0000","last_unstale":"2024-09-15T01:43:56.655181+0000","last_undegraded":"2024-09-15T01:43:56.655181+0000","last_fullsized":"2024-09-15T01:43:56.655181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:58.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":761856,"data_stored":618165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9379999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":757760,"data_stored":617217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":10,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":5,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]}]},{"osd":4,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.899}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":17,"seq":73014444036,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":88289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:01.779 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:44:01.779 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T01:44:01.779 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T01:44:01.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T01:44:02.094 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:02.094 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T01:44:02.107 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T01:44:02.107 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T01:44:02.119 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T01:44:02.119 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:44:02.119 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T01:44:02.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T01:44:02.427 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T01:44:02.428 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:44:02.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:02.715 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:02.718 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:02.731 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T01:44:02.284461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3572344,"kb_used_data":3728,"kb_used_omap":19,"kb_used_meta":3568108,"kb_avail":1128889736,"statfs":{"total":1159641169920,"available":1155983089664,"internally_reserved":0,"allocated":3817472,"data_stored":2126842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3653742624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.893741"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903312+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:43:57.903312+0000","last_peered":"2024-09-15T01:43:57.903312+0000","last_clean":"2024-09-15T01:43:57.903312+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:43:57.903312+0000","last_undegraded":"2024-09-15T01:43:57.903312+0000","last_fullsized":"2024-09-15T01:43:57.903312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416834+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:43:57.416834+0000","last_peered":"2024-09-15T01:43:57.416834+0000","last_clean":"2024-09-15T01:43:57.416834+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:43:57.416834+0000","last_undegraded":"2024-09-15T01:43:57.416834+0000","last_fullsized":"2024-09-15T01:43:57.416834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407762+0000","last_change":"2024-09-15T01:43:57.408162+0000","last_active":"2024-09-15T01:43:57.407762+0000","last_peered":"2024-09-15T01:43:57.407762+0000","last_clean":"2024-09-15T01:43:57.407762+0000","last_became_active":"2024-09-15T01:43:55.500523+0000","last_became_peered":"2024-09-15T01:43:55.500523+0000","last_unstale":"2024-09-15T01:43:57.407762+0000","last_undegraded":"2024-09-15T01:43:57.407762+0000","last_fullsized":"2024-09-15T01:43:57.407762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061773000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416957+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:43:57.416957+0000","last_peered":"2024-09-15T01:43:57.416957+0000","last_clean":"2024-09-15T01:43:57.416957+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:43:57.416957+0000","last_undegraded":"2024-09-15T01:43:57.416957+0000","last_fullsized":"2024-09-15T01:43:57.416957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.415640+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:43:57.415640+0000","last_peered":"2024-09-15T01:43:57.415640+0000","last_clean":"2024-09-15T01:43:57.415640+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:43:57.415640+0000","last_undegraded":"2024-09-15T01:43:57.415640+0000","last_fullsized":"2024-09-15T01:43:57.415640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407639+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:43:57.407639+0000","last_peered":"2024-09-15T01:43:57.407639+0000","last_clean":"2024-09-15T01:43:57.407639+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:43:57.407639+0000","last_undegraded":"2024-09-15T01:43:57.407639+0000","last_fullsized":"2024-09-15T01:43:57.407639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903134+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:43:57.903134+0000","last_peered":"2024-09-15T01:43:57.903134+0000","last_clean":"2024-09-15T01:43:57.903134+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:43:57.903134+0000","last_undegraded":"2024-09-15T01:43:57.903134+0000","last_fullsized":"2024-09-15T01:43:57.903134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.413658+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:43:57.413658+0000","last_peered":"2024-09-15T01:43:57.413658+0000","last_clean":"2024-09-15T01:43:57.413658+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:43:57.413658+0000","last_undegraded":"2024-09-15T01:43:57.413658+0000","last_fullsized":"2024-09-15T01:43:57.413658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":125,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T01:43:56.655181+0000","last_change":"2024-09-15T01:43:54.208642+0000","last_active":"2024-09-15T01:43:56.655181+0000","last_peered":"2024-09-15T01:43:56.655181+0000","last_clean":"2024-09-15T01:43:56.655181+0000","last_became_active":"2024-09-15T01:43:54.206406+0000","last_became_peered":"2024-09-15T01:43:54.206406+0000","last_unstale":"2024-09-15T01:43:56.655181+0000","last_undegraded":"2024-09-15T01:43:56.655181+0000","last_fullsized":"2024-09-15T01:43:56.655181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:58.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":761856,"data_stored":618165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9379999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":757760,"data_stored":617217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":10,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":5,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]}]},{"osd":4,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.899}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":17,"seq":73014444036,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":88289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:02.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:03.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:03.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:03.047 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T01:44:02.284461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3572344,"kb_used_data":3728,"kb_used_omap":19,"kb_used_meta":3568108,"kb_avail":1128889736,"statfs":{"total":1159641169920,"available":1155983089664,"internally_reserved":0,"allocated":3817472,"data_stored":2126842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3653742624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.893741"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903312+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:43:57.903312+0000","last_peered":"2024-09-15T01:43:57.903312+0000","last_clean":"2024-09-15T01:43:57.903312+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:43:57.903312+0000","last_undegraded":"2024-09-15T01:43:57.903312+0000","last_fullsized":"2024-09-15T01:43:57.903312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416834+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:43:57.416834+0000","last_peered":"2024-09-15T01:43:57.416834+0000","last_clean":"2024-09-15T01:43:57.416834+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:43:57.416834+0000","last_undegraded":"2024-09-15T01:43:57.416834+0000","last_fullsized":"2024-09-15T01:43:57.416834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407762+0000","last_change":"2024-09-15T01:43:57.408162+0000","last_active":"2024-09-15T01:43:57.407762+0000","last_peered":"2024-09-15T01:43:57.407762+0000","last_clean":"2024-09-15T01:43:57.407762+0000","last_became_active":"2024-09-15T01:43:55.500523+0000","last_became_peered":"2024-09-15T01:43:55.500523+0000","last_unstale":"2024-09-15T01:43:57.407762+0000","last_undegraded":"2024-09-15T01:43:57.407762+0000","last_fullsized":"2024-09-15T01:43:57.407762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061773000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.416957+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:43:57.416957+0000","last_peered":"2024-09-15T01:43:57.416957+0000","last_clean":"2024-09-15T01:43:57.416957+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:43:57.416957+0000","last_undegraded":"2024-09-15T01:43:57.416957+0000","last_fullsized":"2024-09-15T01:43:57.416957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.415640+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:43:57.415640+0000","last_peered":"2024-09-15T01:43:57.415640+0000","last_clean":"2024-09-15T01:43:57.415640+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:43:57.415640+0000","last_undegraded":"2024-09-15T01:43:57.415640+0000","last_fullsized":"2024-09-15T01:43:57.415640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.407639+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:43:57.407639+0000","last_peered":"2024-09-15T01:43:57.407639+0000","last_clean":"2024-09-15T01:43:57.407639+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:43:57.407639+0000","last_undegraded":"2024-09-15T01:43:57.407639+0000","last_fullsized":"2024-09-15T01:43:57.407639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.903134+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:43:57.903134+0000","last_peered":"2024-09-15T01:43:57.903134+0000","last_clean":"2024-09-15T01:43:57.903134+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:43:57.903134+0000","last_undegraded":"2024-09-15T01:43:57.903134+0000","last_fullsized":"2024-09-15T01:43:57.903134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T01:43:57.413658+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:43:57.413658+0000","last_peered":"2024-09-15T01:43:57.413658+0000","last_clean":"2024-09-15T01:43:57.413658+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:43:57.413658+0000","last_undegraded":"2024-09-15T01:43:57.413658+0000","last_fullsized":"2024-09-15T01:43:57.413658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":125,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T01:43:56.655181+0000","last_change":"2024-09-15T01:43:54.208642+0000","last_active":"2024-09-15T01:43:56.655181+0000","last_peered":"2024-09-15T01:43:56.655181+0000","last_clean":"2024-09-15T01:43:56.655181+0000","last_became_active":"2024-09-15T01:43:54.206406+0000","last_became_peered":"2024-09-15T01:43:54.206406+0000","last_unstale":"2024-09-15T01:43:56.655181+0000","last_undegraded":"2024-09-15T01:43:56.655181+0000","last_fullsized":"2024-09-15T01:43:56.655181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:58.712252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":761856,"data_stored":618165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9379999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":757760,"data_stored":617217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":10,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":5,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]}]},{"osd":4,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":229376,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.899}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":17,"seq":73014444036,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":233472,"data_stored":89256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":88289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:03.049 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:44:03.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T01:44:03.339 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 24 2024-09-15T01:44:03.339 INFO:teuthology.orchestra.run.smithi033.stdout:fsid d386169c-ec95-4076-a6ad-eaab3ccf750b 2024-09-15T01:44:03.339 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-09-15T01:43:41.737992+0000 2024-09-15T01:44:03.339 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-09-15T01:44:01.488775+0000 2024-09-15T01:44:03.339 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 5 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client jewel 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release quincy 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/22087,v1:172.21.15.33:6801/22087] [v2:172.21.15.33:6802/22087,v1:172.21.15.33:6803/22087] exists,up 5bbf5b49-bace-424f-b96c-91c3a25e48e0 2024-09-15T01:44:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6808/22086,v1:172.21.15.33:6809/22086] [v2:172.21.15.33:6810/22086,v1:172.21.15.33:6811/22086] exists,up 3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6816/22088,v1:172.21.15.33:6817/22088] [v2:172.21.15.33:6818/22088,v1:172.21.15.33:6819/22088] exists,up fcc939b7-c3fb-4117-af41-9eec226f3020 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6824/22089,v1:172.21.15.33:6825/22089] [v2:172.21.15.33:6826/22089,v1:172.21.15.33:6827/22089] exists,up 57f1d668-f85b-4b0c-9093-fb5fef4b91ef 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6800/22096,v1:172.21.15.47:6801/22096] [v2:172.21.15.47:6802/22096,v1:172.21.15.47:6803/22096] exists,up a274d12c-8683-491d-8656-5af5d12164ee 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6816/22097,v1:172.21.15.47:6817/22097] [v2:172.21.15.47:6818/22097,v1:172.21.15.47:6819/22097] exists,up f0a90a47-9b16-46f6-a2a7-afa4a6f2a298 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6808/22108,v1:172.21.15.47:6809/22108] [v2:172.21.15.47:6810/22108,v1:172.21.15.47:6811/22108] exists,up 194e23d1-7f52-43ee-8557-52d75a3c22dc 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6824/22109,v1:172.21.15.47:6825/22109] [v2:172.21.15.47:6826/22109,v1:172.21.15.47:6827/22109] exists,up 1d1c61e7-be03-4c72-bc95-e187eb95a1b8 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6801/22357,v1:172.21.15.155:6803/22357] [v2:172.21.15.155:6805/22357,v1:172.21.15.155:6807/22357] exists,up 1f23463d-2214-416b-b3ba-23132f2d84ee 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/22359,v1:172.21.15.155:6802/22359] [v2:172.21.15.155:6804/22359,v1:172.21.15.155:6806/22359] exists,up 39226fc7-a654-41c6-9d70-5ea2f69c236b 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/22358,v1:172.21.15.155:6817/22358] [v2:172.21.15.155:6818/22358,v1:172.21.15.155:6819/22358] exists,up eaafc17e-407d-43aa-8083-2c56660d7db4 2024-09-15T01:44:03.341 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/22360,v1:172.21.15.155:6825/22360] [v2:172.21.15.155:6826/22360,v1:172.21.15.155:6827/22360] exists,up 05fdd01d-8c78-4a61-81e7-1d9816e951a3 2024-09-15T01:44:03.353 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/22087,v1:172.21.15.33:6801/22087] [v2:172.21.15.33:6802/22087,v1:172.21.15.33:6803/22087] exists,up 5bbf5b49-bace-424f-b96c-91c3a25e48e0', 'osd.1 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6808/22086,v1:172.21.15.33:6809/22086] [v2:172.21.15.33:6810/22086,v1:172.21.15.33:6811/22086] exists,up 3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464', 'osd.2 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6816/22088,v1:172.21.15.33:6817/22088] [v2:172.21.15.33:6818/22088,v1:172.21.15.33:6819/22088] exists,up fcc939b7-c3fb-4117-af41-9eec226f3020', 'osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6824/22089,v1:172.21.15.33:6825/22089] [v2:172.21.15.33:6826/22089,v1:172.21.15.33:6827/22089] exists,up 57f1d668-f85b-4b0c-9093-fb5fef4b91ef', 'osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6800/22096,v1:172.21.15.47:6801/22096] [v2:172.21.15.47:6802/22096,v1:172.21.15.47:6803/22096] exists,up a274d12c-8683-491d-8656-5af5d12164ee', 'osd.5 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6816/22097,v1:172.21.15.47:6817/22097] [v2:172.21.15.47:6818/22097,v1:172.21.15.47:6819/22097] exists,up f0a90a47-9b16-46f6-a2a7-afa4a6f2a298', 'osd.6 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6808/22108,v1:172.21.15.47:6809/22108] [v2:172.21.15.47:6810/22108,v1:172.21.15.47:6811/22108] exists,up 194e23d1-7f52-43ee-8557-52d75a3c22dc', 'osd.7 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6824/22109,v1:172.21.15.47:6825/22109] [v2:172.21.15.47:6826/22109,v1:172.21.15.47:6827/22109] exists,up 1d1c61e7-be03-4c72-bc95-e187eb95a1b8', 'osd.8 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6801/22357,v1:172.21.15.155:6803/22357] [v2:172.21.15.155:6805/22357,v1:172.21.15.155:6807/22357] exists,up 1f23463d-2214-416b-b3ba-23132f2d84ee', 'osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/22359,v1:172.21.15.155:6802/22359] [v2:172.21.15.155:6804/22359,v1:172.21.15.155:6806/22359] exists,up 39226fc7-a654-41c6-9d70-5ea2f69c236b', 'osd.10 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/22358,v1:172.21.15.155:6817/22358] [v2:172.21.15.155:6818/22358,v1:172.21.15.155:6819/22358] exists,up eaafc17e-407d-43aa-8083-2c56660d7db4', 'osd.11 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/22360,v1:172.21.15.155:6825/22360] [v2:172.21.15.155:6826/22360,v1:172.21.15.155:6827/22360] exists,up 05fdd01d-8c78-4a61-81e7-1d9816e951a3'] 2024-09-15T01:44:03.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T01:44:03.448 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:03.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mon.a", 2024-09-15T01:44:03.448 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster": "ceph", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "admin_socket_mode": "", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_client_required": "cephx, none", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_cluster_required": "cephx", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_debug": "true", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_service_required": "cephx", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_supported": "cephx", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_aio": "true", 2024-09-15T01:44:03.449 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_aio_reap_max": "16", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_async_discard": "false", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_block_size": "4096", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_debug_aio": "false", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T01:44:03.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_enable_discard": "false", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_flock_retry": "3", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_inject_crash": "0", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_ioring": "false", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_ioring_hipri": "false", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bdev_type": "", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T01:44:03.451 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_allocator": "hybrid", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_buffered_io": "true", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T01:44:03.452 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_replay_recovery": "false", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_sync_write": "false", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_allocator": "hybrid", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_async_db_compaction": "true", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T01:44:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_create": "true", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_db_create": "false", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_db_path": "", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_db_size": "0", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_path": "", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T01:44:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_size": "107374182400", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_wal_create": "false", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_wal_path": "", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bluefs": "true", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:44:03.455 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_autotune": "true", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_size": "0", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T01:44:03.456 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_type": "2q", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_clone_cow": "true", 2024-09-15T01:44:03.457 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T01:44:03.458 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_mode": "none", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_freelist": "false", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T01:44:03.459 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T01:44:03.460 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T01:44:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T01:44:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T01:44:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T01:44:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T01:44:03.463 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fail_eio": "false", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T01:44:03.464 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T01:44:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_blob_size": "0", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T01:44:03.466 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T01:44:03.467 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_spdk_mem": "512", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_tracing": "false", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T01:44:03.468 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T01:44:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_cluster_require_version": "2", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_require_signatures": "false", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_require_version": "2", 2024-09-15T01:44:03.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_service_require_signatures": "false", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_service_require_version": "2", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "cephx_sign_messages": "true", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "chdir": "", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_acl_type": "", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_asio_thread_count": "2", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_cache_mid": "0.750000", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_cache_size": "16384", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_caps_release_delay": "5", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_check_pool_perm": "true", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_debug_force_sync_read": "false", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_debug_getattr_caps": "false", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_debug_inject_features": "", 2024-09-15T01:44:03.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_die_on_failed_remount": "false", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_dirsize_rbytes": "true", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_force_lazyio": "false", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_fs": "", 2024-09-15T01:44:03.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_inject_release_failure": "false", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_max_inline_size": "4096", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_mds_namespace": "", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_metadata": "", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_mount_gid": "-1", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_mount_timeout": "300", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_mount_uid": "-1", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_mountpoint": "/", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_notify_timeout": "10", 2024-09-15T01:44:03.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc": "true", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc_max_objects": "1000", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc_size": "209715200", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_permissions": "true", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_quota": "true", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_quota_df": "true", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_readahead_max_bytes": "0", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_readahead_max_periods": "4", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_readahead_min": "131072", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_reconnect_stale": "false", 2024-09-15T01:44:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "client_shutdown_timeout": "30", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_snapdir": ".snap", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_tick_interval": "1", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_trace": "", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_use_faked_inos": "false", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_use_random_mds": "false", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_graylog": "false", 2024-09-15T01:44:03.475 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_graylog_port": "12201", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_monitors": "default=true", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_syslog": "false", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "clog_to_syslog_level": "info", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "-", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_network": "", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_network_interface": "", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "compressor_zlib_isal": "false", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "compressor_zlib_level": "5", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "compressor_zstd_level": "1", 2024-09-15T01:44:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location_hook": "", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location_hook_timeout": "10", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "daemonize": "false", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_alienstore": "0/5", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T01:44:03.477 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_asok": "1/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_asok_assert_abort": "false", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_auth": "20/20", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_bdev": "1/3", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_bluefs": "1/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_bluestore": "1/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_buffer": "0/1", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_cephsqlite": "0/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_civetweb": "1/10", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_client": "0/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_compressor": "1/5", 2024-09-15T01:44:03.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_context": "0/1", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_crush": "1/1", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_crypto": "1/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_dpdk": "1/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_eventtrace": "1/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_filer": "0/1", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_filestore": "1/3", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_finisher": "1/1", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_fuse": "1/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T01:44:03.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_javaclient": "1/5", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_journal": "1/3", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_journaler": "0/5", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_kstore": "1/5", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_leveldb": "4/5", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_lockdep": "0/1", 2024-09-15T01:44:03.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mclock": "1/5", 2024-09-15T01:44:03.481 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds": "1/5", 2024-09-15T01:44:03.481 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds_balancer": "1/5", 2024-09-15T01:44:03.481 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds_locker": "1/5", 2024-09-15T01:44:03.481 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds_log": "1/5", 2024-09-15T01:44:03.481 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mds_migrator": "1/5", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_memdb": "4/5", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mgr": "2/5", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mgrc": "1/5", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_mon": "20/20", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_monc": "0/10", 2024-09-15T01:44:03.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_ms": "1/1", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_none": "0/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_objclass": "0/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_objectcacher": "0/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_objecter": "0/1", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_optracker": "0/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_osd": "1/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_paxos": "20/20", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_perfcounter": "1/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_prioritycache": "1/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rados": "0/5", 2024-09-15T01:44:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rbd": "0/5", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rbd_replay": "0/5", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_refs": "0/0", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_reserver": "1/1", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rgw": "1/5", 2024-09-15T01:44:03.484 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rgw_sync": "1/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_rocksdb": "4/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_cache": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_device": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_journal": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_lba": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_odata": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_omap": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_onode": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_seastore_tm": "0/5", 2024-09-15T01:44:03.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_striper": "0/1", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_test": "0/5", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_throttle": "1/1", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_timer": "0/1", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug_tp": "0/5", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "device_failure_prediction_mode": "none", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "err_to_graylog": "false", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "err_to_journald": "false", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "err_to_stderr": "true", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "err_to_syslog": "false", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event_tracing": "false", 2024-09-15T01:44:03.486 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_http_port": "9926", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_prio_limit": "5", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_sort_metrics": "true", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "exporter_stats_period": "5", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "fake_statfs_for_testing": "0", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "fatal_signal_handlers": "true", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "filer_max_purge_ops": "10", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "filer_max_truncate_ops": "128", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_blackhole": "false", 2024-09-15T01:44:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_btrfs_snap": "true", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_caller_concurrency": "10", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_debug_omap_check": "false", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_debug_verify_split": "false", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_dump_file": "", 2024-09-15T01:44:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fadvise": "true", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fail_eio": "true", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fd_cache_size": "128", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fiemap": "false", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_inject_stall": "0", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_journal_parallel": "false", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_journal_trailing": "false", 2024-09-15T01:44:03.489 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_journal_writeahead": "false", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_kill_at": "0", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T01:44:03.490 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_merge_threshold": "-10", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_odsync_write": "false", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_omap_backend_path": "", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_op_threads": "2", 2024-09-15T01:44:03.491 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_punch_hole": "false", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_queue_max_ops": "50", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T01:44:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_seek_data_hole": "false", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_sloppy_crc": "false", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_splice": "false", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_split_multiple": "2", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_split_rand_factor": "20", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_update_to": "1000", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T01:44:03.493 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_xfs_extsize": "false", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "filestore_zfs_snap": "false", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "fio_dir": "/tmp/fio", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "d386169c-ec95-4076-a6ad-eaab3ccf750b", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_allow_other": "true", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T01:44:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_big_writes": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_debug": "false", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_default_permissions": "false", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_disable_pagecache": "false", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_max_write": "0", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_multithreaded": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_require_active_mds": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_set_user_groups": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_splice_move": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_splice_read": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_splice_write": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T01:44:03.495 INFO:teuthology.orchestra.run.smithi033.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "gss_target_name": "ceph", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_file": "", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_inject_failure": "0", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_interval": "5", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "host": "smithi033", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T01:44:03.496 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "inject_early_sigterm": "false", 2024-09-15T01:44:03.497 INFO:teuthology.orchestra.run.smithi033.stdout: "jaeger_tracing_enable": "false", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_aio": "true", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_align_min_size": "65536", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_block_align": "true", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_block_size": "4096", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_dio": "true", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_discard": "false", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_force_aio": "false", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_ignore_corruption": "false", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_max_write_entries": "100", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_replay_from": "0", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T01:44:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_write_header_frequency": "0", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journal_zero_on_create": "false", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journaler_prefetch_periods": "10", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journaler_prezero_periods": "5", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "journaler_write_head_interval": "15", 2024-09-15T01:44:03.499 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "keyfile": "", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_backend": "rocksdb", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_max_bytes": "67108864", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_max_ops": "512", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T01:44:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_onode_map_size": "1024", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "kstore_sync_transaction": "false", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_block_size": "65536", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_bloom_size": "0", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_cache_size": "536870912", 2024-09-15T01:44:03.501 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_compact_on_mount": "false", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_compression": "false", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_log": "", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_max_open_files": "0", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_paranoid": "false", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "librados_thread_count": "2", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "lockdep": "false", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "lockdep_force_backtrace": "false", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "log_coarse_timestamps": "true", 2024-09-15T01:44:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_flush_on_exit": "false", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_graylog_port": "12201", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_max_new": "1000", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_max_recent": "10000", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_stderr_prefix": "", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": "true", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_graylog": "false", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_journald": "false", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_stderr": "false", 2024-09-15T01:44:03.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_syslog": "false", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_action_on_write_error": "1", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_alternate_name_max": "8192", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_asio_thread_count": "2", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_export_pin": "true", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T01:44:03.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_interval": "10", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_max": "-1", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_max_until": "-1", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_merge_size": "50", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_mode": "0", 2024-09-15T01:44:03.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_split_bits": "3", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_split_size": "10000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T01:44:03.506 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_mid": "0.700000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_trim_interval": "1", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T01:44:03.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_debug_auth_pins": "false", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_debug_frag": "false", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_debug_scatterstat": "false", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_debug_subtrees": "false", 2024-09-15T01:44:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_default_dir_hash": "2", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_defer_session_stale": "true", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dir_max_entries": "0", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_early_reply": "true", 2024-09-15T01:44:03.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_enable_op_tracker": "true", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_enforce_unique_name": "true", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T01:44:03.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_health_dummy": "false", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_join_fs": "", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_journal_format": "1", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_create_at": "0", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_export_at": "0", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_import_at": "0", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_journal_at": "0", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T01:44:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_link_at": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_openc_at": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_rename_at": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_max_events": "-1", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_max_segments": "128", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_pause": "false", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_segment_size": "0", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T01:44:03.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_completed_requests": "100000", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_export_size": "20971520", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_file_recover": "32", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_purge_files": "64", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_purge_ops": "8192", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_metrics_update_interval": "2", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_min_caps_per_client": "100", 2024-09-15T01:44:03.513 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_numa_node": "-1", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_history_duration": "600", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_history_size": "20", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_op_log_threshold": "5", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_ping_grace": "15", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_ping_interval": "5", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T01:44:03.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_max_caps": "30000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_replay_interval": "1.000000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_root_ino_gid": "0", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_root_ino_uid": "0", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T01:44:03.515 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_shutdown_check": "0", 2024-09-15T01:44:03.516 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_skip_ino": "0", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_snap_min_uid": "0", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_snap_rstat": "false", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_symlink_recovery": "true", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_thrash_exports": "0", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_thrash_fragments": "0", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_tick_interval": "5.000000", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_valgrind_exit": "false", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_verify_backtrace": "1", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_verify_scatter": "false", 2024-09-15T01:44:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_wipe_sessions": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool_debug": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "memstore_page_set": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "memstore_page_size": "65536", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_client_bytes": "134217728", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_client_messages": "512", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T01:44:03.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_mds_messages": "128", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_mon_messages": "128", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_osd_messages": "8192", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_pool": "true", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_standby_modules": "true", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_stats_period": "5", 2024-09-15T01:44:03.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_stats_threshold": "5", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_tick_period": "2", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_allow_pool_delete": "true", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_bytes": "104857600", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_directed_command_retry": "5", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T01:44:03.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_level": "debug", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T01:44:03.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_compact_on_start": "false", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_compact_on_trim": "true", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_cpu_threads": "4", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T01:44:03.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_data_avail_crit": "5", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_data_avail_warn": "5", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_dump_json": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_extra_checks": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_no_require_pacific": "false", 2024-09-15T01:44:03.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_no_require_quincy": "false", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_election_default_strategy": "1", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_election_timeout": "5.000000", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_enable_op_tracker": "true", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_fake_pool_delete": "false", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_force_quorum_join": "false", 2024-09-15T01:44:03.524 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_log_update_period": "5", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_max_detail": "50", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_to_clog": "true", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_host": "172.21.15.33,172.21.15.47,172.21.15.155", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_host_override": "", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_initial_members": "", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T01:44:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_lease": "5.000000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_log_full_interval": "50", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_log_max": "10000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_log_max_summary": "50", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_log_epochs": "500", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_osd": "10000", 2024-09-15T01:44:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_memory_autotune": "true", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_memory_target": "2147483648", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_digest_period": "5", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T01:44:03.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_complaint_time": "30", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_history_duration": "600", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_history_size": "20", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_op_log_threshold": "5", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T01:44:03.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_cache_size": "500", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T01:44:03.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_report_timeout": "900", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T01:44:03.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T01:44:03.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_reweight_max_osds": "4", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_scrub_interval": "300", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_scrub_max_keys": "100", 2024-09-15T01:44:03.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_scrub_timeout": "300", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_session_timeout": "300", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_smart_report_timeout": "5", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_stretch_pool_size": "4", 2024-09-15T01:44:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_debug": "false", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T01:44:03.534 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_tick_interval": "5", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T01:44:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_older_version": "true", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T01:44:03.536 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": "", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_op_threads": "3", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_cm": "false", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_device_name": "", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T01:44:03.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_sl": "3", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_rdma_type": "ib", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_async_reap_threshold": "5", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_before_connect": "false", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_ipv4": "true", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_ipv6": "false", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_msgr1": "true", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_msgr2": "true", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_port_max": "7568", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_port_min": "6800", 2024-09-15T01:44:03.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_retry_count": "3", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_bind_retry_delay": "5", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_blackhole_client": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_blackhole_mds": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_blackhole_mgr": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_blackhole_mon": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_blackhole_osd": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_client_mode": "crc secure", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_cluster_type": "", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_compress_secure": "false", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T01:44:03.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_crc_data": "true", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_crc_header": "true", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_die_on_bug": "true", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_die_on_old_message": "true", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T01:44:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_hugepages": "", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_lro": "true", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_pmd": "", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_port_id": "0", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T01:44:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_dump_on_send": "false", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_inject_delay_type": "", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_inject_socket_failures": "2500", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_max_accept_failures": "4", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_max_backoff": "15.000000", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T01:44:03.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_osd_compress_mode": "none", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_pq_min_cost": "65536", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_public_type": "", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_service_mode": "crc secure", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_tcp_nodelay": "true", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ms_type": "async+posix", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "no_config_file": "false", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T01:44:03.543 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_inflight_ops": "1024", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter_timeout": "10.000000", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "objectstore_blackhole": "false", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "openssl_engine_opts": "", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_max_ops": "4", 2024-09-15T01:44:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_agent_slop": "0.020000", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_asio_thread_count": "2", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_async_recovery_min_cost": "1", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backfill_scan_max": "512", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backfill_scan_min": "64", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backoff_on_peering": "false", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T01:44:03.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_beacon_report_interval": "300", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bench_duration": "30", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_blkin_trace_all": "false", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T01:44:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_class_update_on_start": "true", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_client_message_cap": "256", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_client_op_priority": "63", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_client_watch_timeout": "30", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_command_max_records": "256", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_compact_on_start": "false", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_crush_update_on_start": "true", 2024-09-15T01:44:03.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T01:44:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_op_order": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_shutdown": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T01:44:03.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_default_notify_timeout": "30", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_enable_op_tracker": "true", 2024-09-15T01:44:03.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_fast_info": "true", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_fast_shutdown": "true", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_function_tracing": "false", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_grace": "20", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_interval": "6", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T01:44:03.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_stale": "600", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_journal_size": "5120", 2024-09-15T01:44:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_kill_backfill_at": "0", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_map_cache_size": "50", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_map_dedup": "true", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_map_message_max": "40", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_attr_name_len": "100", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_attr_size": "0", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_backfills": "1", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_markdown_count": "5", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_markdown_period": "600", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_object_name_len": "2048", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T01:44:03.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_object_size": "134217728", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_pgls": "1024", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_push_cost": "8388608", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_push_objects": "10", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_scrubs": "3", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T01:44:03.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_max_write_size": "90", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_profile": "balanced", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T01:44:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_base": "805306368", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_target": "4294967296", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_target_autotune": "false", 2024-09-15T01:44:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_min_recovery_priority": "0", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mon_report_interval": "5", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_num_cache_shards": "32", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_numa_node": "-1", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T01:44:03.557 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_objecter_finishers": "1", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_objectstore": "bluestore", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_objectstore_fuse": "false", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_objectstore_tracing": "false", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_history_duration": "600", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_history_size": "20", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_log_threshold": "5", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_shards": "0", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T01:44:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_op_thread_timeout": "15", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_open_classes_on_start": "true", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_os_flags": "0", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_peering_op_priority": "255", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_delete_priority": "5", 2024-09-15T01:44:03.559 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T01:44:03.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T01:44:03.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_flags": "0", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_min_size": "0", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_size": "2", 2024-09-15T01:44:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_default_type": "replicated", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_cost": "20971520", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_active": "0", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T01:44:03.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_op_priority": "3", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_priority": "5", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T01:44:03.564 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_repair_during_recovery": "false", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T01:44:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_cost": "52428800", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_end_hour": "0", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T01:44:03.566 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_priority": "5", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_skip_data_digest": "false", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_smart_report_timeout": "5", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_priority": "5", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T01:44:03.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_target_transaction_size": "30", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T01:44:03.568 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_tracing": "false", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_use_stale_snap": "false", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_kill_at": "0", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_max_join_drift": "10", 2024-09-15T01:44:03.569 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_min": "500", 2024-09-15T01:44:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_min_wait": "0.050000", 2024-09-15T01:44:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T01:44:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_service_trim_max": "500", 2024-09-15T01:44:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T01:44:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_service_trim_min": "250", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_stash_full_interval": "25", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_trim_max": "500", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "paxos_trim_min": "250", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "perf": "true", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "-", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrv": "", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "public_bind_addr": "-", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "public_network": "", 2024-09-15T01:44:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: "public_network_interface": "", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "qat_compressor_enabled": "false", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rados_mon_op_timeout": "0", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rados_osd_op_timeout": "0", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rados_tracing": "false", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_atime_update_interval": "60", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache": "true", 2024-09-15T01:44:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_policy": "writearound", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_size": "33554432", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_compression_hint": "none", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_clone_format": "auto", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_data_pool": "", 2024-09-15T01:44:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_features": "61", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_format": "2", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_map_options": "", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_order": "22", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_pool": "rbd", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_stripe_count": "0", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T01:44:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_io_scheduler": "simple", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_order": "24", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_pool": "", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_journal_splay_width": "4", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T01:44:03.575 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T01:44:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T01:44:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_op_threads": "1", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T01:44:03.578 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_plugins": "", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T01:44:03.579 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T01:44:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_tracing": "false", 2024-09-15T01:44:03.581 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_validate_names": "true", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_validate_pool": "true", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "restapi_base_url": "", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "restapi_log_level": "", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_admin_entry": "admin", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_backend_store": "rados", 2024-09-15T01:44:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_barbican_url": "", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_beast_enable_async": "true", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_cache_enabled": "true", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_cache_lru_size": "10000", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_content_length_compat": "false", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T01:44:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_cross_domain_policy": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T01:44:03.584 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T01:44:03.585 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T01:44:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T01:44:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data_log_window": "30", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T01:44:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T01:44:03.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dns_name": "", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dns_s3website_name": "", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_ops_log": "false", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T01:44:03.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_static_website": "false", 2024-09-15T01:44:03.591 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enable_usage_log": "false", 2024-09-15T01:44:03.591 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T01:44:03.591 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T01:44:03.591 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_expose_bucket": "false", 2024-09-15T01:44:03.591 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_extended_http_attrs": "", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_objs": "32", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T01:44:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_init_timeout": "300", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T01:44:03.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_password": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_project": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_token": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_admin_user": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_api_version": "2", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T01:44:03.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_url": "", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_max_objs": "32", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_max_worker": "3", 2024-09-15T01:44:03.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ldap_uri": "", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_log_http_headers": "", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T01:44:03.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_attr_size": "0", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_listing_results": "1000", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_notify_retries": "10", 2024-09-15T01:44:03.597 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T01:44:03.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T01:44:03.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_num_control_oids": "8", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_numa_node": "-1", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_op_tracing": "false", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_opa_token": "", 2024-09-15T01:44:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_opa_url": "", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ops_log_file_path": "", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ops_log_rados": "true", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T01:44:03.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_print_continue": "true", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_rados_tracing": "false", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_realm": "", 2024-09-15T01:44:03.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_realm_id": "", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_region": "", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_request_uri": "", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T01:44:03.603 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_resolve_cname": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_run_sync_thread": "true", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T01:44:03.604 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_script_uri": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_service_provider_name": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_client_id": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_client_secret": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_entry": "sts", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_key": "sts", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_auth_url": "", 2024-09-15T01:44:03.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_custom_header": "", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_need_stats": "true", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_tenant_name": "", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_url": "", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_lease_period": "120", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T01:44:03.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_thread_pool_size": "512", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_comment": "", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_createby": "", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_encoding": "", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_flag": "false", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_origin": "", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T01:44:03.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_torrent_tracker": "", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_usage_max_shards": "32", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_use_opa_authz": "false", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T01:44:03.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_user_unique_email": "true", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_verify_ssl": "true", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zone": "", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zone_id": "", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zonegroup": "", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zonegroup_id": "", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_block_size": "4096", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T01:44:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T01:44:03.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_partition_filters": "false", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_perf": "false", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "run_dir": "/var/run/ceph", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_block_create": "true", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_device_size": "107374182400", 2024-09-15T01:44:03.611 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "seastore_segment_size": "67108864", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "setgroup": "", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "setuser": "", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "setuser_match_path": "", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "thp": "false", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "threadpool_default_timeout": "60", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T01:44:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "throttler_perf_counter": "true" 2024-09-15T01:44:03.613 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:03.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T01:44:03.753 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a: {} 2024-09-15T01:44:03.768 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b: {} 2024-09-15T01:44:03.783 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c: {} 2024-09-15T01:44:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:44:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:44:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:44:03.799 DEBUG:teuthology.orchestra.run.smithi155:> type ceph-objectstore-tool 2024-09-15T01:44:03.806 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:44:03.806 DEBUG:teuthology.orchestra.run.smithi047:> type ceph-objectstore-tool 2024-09-15T01:44:03.812 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:44:03.813 DEBUG:teuthology.orchestra.run.smithi033:> type ceph-objectstore-tool 2024-09-15T01:44:03.818 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T01:44:03.819 INFO:teuthology.run_tasks:Running task rados... 2024-09-15T01:44:03.832 INFO:tasks.rados:Beginning rados... 2024-09-15T01:44:03.833 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T01:44:03.833 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:44:03.833 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T01:44:03.833 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-15T01:44:03.858 INFO:tasks.rados:joining rados 2024-09-15T01:44:03.858 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T01:44:03.858 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:03.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:03.859 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T01:44:03.859 INFO:tasks.thrashosds.thrasher:Setting osd 8 primary_affinity to 0.000000 2024-09-15T01:44:03.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 0 2024-09-15T01:44:03.859 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T01:44:03.860 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T01:44:03.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:03.860 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T01:44:03.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:03.861 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T01:44:03.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:44:03.861 INFO:tasks.rados:clients are ['client.0'] 2024-09-15T01:44:03.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set lrcprofile crush-failure-domain=osd k=4 l=3 m=2 name=lrcprofile plugin=lrc 2024-09-15T01:44:03.862 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:03.856+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:03.962 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:03.956+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:03.983 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:03.983 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:03.983 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:03.983 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:03.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:04.043 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:04.062 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:04.062 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:04.056+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:04.079 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:04.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:04.105 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.106 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:04.148 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:04.164 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:04.156+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:04.172 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:04.193 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:04.216 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:04.231 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.231 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.239 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:04.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:04.262 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:04.264 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:04.259+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:04.293 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:04.295 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:04.295 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:04.296 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:04.361 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.361 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.365 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:04.359+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:04.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:04.463 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.463 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.465 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:04.459+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:04.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.8 primary-affinity to 0 (802) 2024-09-15T01:44:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:44:04.538 INFO:tasks.rados:starting run 0 out of 1 2024-09-15T01:44:04.538 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-15T01:44:04.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure lrcprofile 2024-09-15T01:44:04.566 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:04.559+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:04.594 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.595 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:04.667 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:04.659+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:04.697 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.697 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:04.768 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:04.760+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:04.808 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.808 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:04.869 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:04.864+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:04.907 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:04.908 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:04.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:04.970 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:04.963+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:05.011 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:05.073 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:05.063+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:05.118 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.118 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:05.171 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:05.164+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:05.225 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.225 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:05.281 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:05.276+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:05.348 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.348 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:05.372 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:05.368+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:05.457 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.457 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:05.473 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:05.468+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:05.565 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.565 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.574 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:05.574 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:05.568+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:05.672 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.672 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:05.668+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:05.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:05.776 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:05.767+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:05.778 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.778 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:05.877 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:05.871+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:05.889 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.890 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:05.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:05.978 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:05.975+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:05.998 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:05.998 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:06.079 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:06.076+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:06.108 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:06.108 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:06.180 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:06.172+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:06.217 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:06.218 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:06.281 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:06.276+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:06.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:06.380 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:06.380 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.381 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:06.376+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:06.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:06.461 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:06.482 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:06.476+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:06.483 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:06.506 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:06.528 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:06.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:44:06.544 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'unique_pool_0' created 2024-09-15T01:44:06.555 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:06.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-15T01:44:06.583 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:06.576+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:06.612 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:06.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:06.633 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:06.658 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:06.684 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:06.676+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:06.694 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:06.727 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:06.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:06.728 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:06.728 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:06.729 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:06.737 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:06.761 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:06.784 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:06.780+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:06.840 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:06.840 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:06.840 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:06.840 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:06.840 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:06.849 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:06.885 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:06.880+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:06.948 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:06.949 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:06.949 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:06.949 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [] 2024-09-15T01:44:06.949 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:06.957 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:06.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:06.980+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:07.058 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.059 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.059 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:07.059 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.068 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:07.087 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:07.080+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:07.165 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.165 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.165 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:07.166 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:07.166 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.174 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:07.188 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:07.180+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:07.273 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.274 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:07.274 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:07.274 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [] 2024-09-15T01:44:07.274 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:07.288 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:07.283+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:07.387 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.387 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.387 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:07.388 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.390 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:07.383+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:07.396 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:07.490 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:07.484+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:07.541 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.541 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:07.541 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:07.541 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.554 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:07.558 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T01:44:07.560 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:44:07.591 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:07.583+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:07.648 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.648 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:07.648 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:07.648 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [] 2024-09-15T01:44:07.648 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.657 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:07.692 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:07.688+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:07.756 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.756 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.756 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:07.757 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.765 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:07.793 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:07.788+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:07.862 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.863 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:07.863 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:07.863 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:07.863 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:07.872 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:07.894 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:07.888+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.978 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.724635651, 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027436790000000002, 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:07.979 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248574+0000", 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.327e-06 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248717+0000", 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014298300000000001 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.980 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248751+0000", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3615999999999998e-05 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248761+0000", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0346000000000001e-05 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248855+0000", 2024-09-15T01:44:07.981 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3807999999999994e-05 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248890+0000", 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.5099999999999999e-05 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.251228+0000", 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0023381169999999998 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:07.982 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.251317+0000", 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8381999999999995e-05 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.531227354, 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:07.983 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.984 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:07.985 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.986 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:07.987 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.530265827000001, 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:07.988 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.989 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:07.990 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.991 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.992 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.527700587, 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.993 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:07.994 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:07.995 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.996 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.526739094, 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.998 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:07.999 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.000 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:08.001 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.524656535, 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:08.002 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.003 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.005 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:08.006 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.523625198, 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.007 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.008 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:08.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.010 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.522694649, 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.011 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.012 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:08.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.014 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.521839241, 2024-09-15T01:44:08.016 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.017 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.018 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.019 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:08.020 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:50 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'32, mlcod=20'25)", 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.520912990999999, 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00272172, 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.021 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452297+0000", 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5030000000000001e-06 2024-09-15T01:44:08.022 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452465+0000", 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00016775000000000001 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452496+0000", 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1312000000000001e-05 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.023 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452508+0000", 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1873e-05 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452580+0000", 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2071999999999997e-05 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452624+0000", 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3936000000000001e-05 2024-09-15T01:44:08.024 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454942+0000", 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0023181880000000001 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.455017+0000", 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.5086000000000006e-05 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.025 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.519322819999999, 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.026 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.027 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:08.029 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:08.030 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.967283108, 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.031 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.032 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.033 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.034 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.965868884000001, 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.035 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:08.036 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.037 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.038 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.039 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.964862862, 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.040 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:08.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.042 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.043 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.963592234, 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:08.044 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.045 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:08.046 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.047 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.048 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:76 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=20'48)", 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.962616418, 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029957009999999999, 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.049 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010593+0000", 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.050 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010606+0000", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3315999999999999e-05 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010621+0000", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4355000000000001e-05 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010626+0000", 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6840000000000001e-06 2024-09-15T01:44:08.051 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012455+0000", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0018292650000000001 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012507+0000", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1099e-05 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013530+0000", 2024-09-15T01:44:08.052 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00102333 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013588+0000", 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7886000000000001e-05 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:99 1.0 e21/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'73, mlcod=21'68)", 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:08.053 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.414705487999999, 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029651880000000001, 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:08.054 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558504+0000", 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559396+0000", 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00089181999999999998 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.055 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559422+0000", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6482000000000002e-05 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559435+0000", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3107e-05 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559522+0000", 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6501000000000004e-05 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559562+0000", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0209000000000001e-05 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.561405+0000", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001842865 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.561468+0000", 2024-09-15T01:44:08.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3338999999999998e-05 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:101 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'75, mlcod=21'73)", 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.408782762, 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027461389999999999, 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.058 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.059 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564427+0000", 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3370000000000001e-06 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565185+0000", 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00075745100000000002 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565212+0000", 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7042e-05 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565227+0000", 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5497000000000001e-05 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565350+0000", 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00012340300000000001 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.061 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565397+0000", 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567111+0000", 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001713715 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567172+0000", 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.0819000000000002e-05 2024-09-15T01:44:08.062 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.407894153999999, 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.063 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.064 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:08.065 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:103 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'77, mlcod=21'73)", 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:08.067 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 12.406905491, 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.002916709, 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:08.068 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566304+0000", 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.1399999999999997e-07 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567195+0000", 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00089143999999999996 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567219+0000", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.4202999999999998e-05 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567246+0000", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6356e-05 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567359+0000", 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00011372800000000001 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567407+0000", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.7741999999999998e-05 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569144+0000", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001736958 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569220+0000", 2024-09-15T01:44:08.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.5667999999999994e-05 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:08.072 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:08.073 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:08.067+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:08.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:08.175 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:08.171+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:08.184 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.185 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:08.185 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:08.185 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:08.275 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:08.271+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:08.294 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:08.294 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:08.376 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:08.368+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:08.413 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.413 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:08.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:08.413 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [] 2024-09-15T01:44:08.413 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:08.477 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:08.472+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:08.526 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.526 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:08.527 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:08.527 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:08.573 DEBUG:teuthology.orchestra.run.smithi033:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-15T01:44:08.578 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:08.572+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight read -> 100 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight delete -> 50 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight snap_create -> 50 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight snap_remove -> 50 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight rollback -> 50 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight setattr -> 25 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight rmattr -> 25 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight copy_from -> 50 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:adding op weight append -> 100 2024-09-15T01:44:08.592 INFO:tasks.rados.rados.0.smithi033.stdout:ceph version 17.2.7-1643-gd75f2382 (d75f23820806da622f561ee5ee7551015285993f) quincy (stable) 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout:Configuration: 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Number of operations: 400 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Number of objects: 50 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Max in flight operations: 16 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Object size (in bytes): 4000000 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Write stride min: 400000 2024-09-15T01:44:08.593 INFO:tasks.rados.rados.0.smithi033.stdout: Write stride max: 800000 2024-09-15T01:44:08.637 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.637 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:08.637 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:08.637 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:08.637 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:08.678 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:08.675+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:08.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:08.779 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:08.772+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:08.792 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:08.792 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:08.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4317.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:43:56.398252+0000", 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.391700866000001, 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001788013, 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398252+0000", 2024-09-15T01:44:08.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398252+0000", 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398253+0000", 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2299999999999999e-07 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398266+0000", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3213e-05 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398308+0000", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1764999999999997e-05 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398337+0000", 2024-09-15T01:44:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9026999999999998e-05 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398500+0000", 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000163256 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.398548+0000", 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7292000000000001e-05 2024-09-15T01:44:08.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.399974+0000", 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014265230000000001 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:56.400040+0000", 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6013999999999995e-05 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:08.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4317.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:43:57.413786+0000", 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.376166509000001, 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001561703, 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413786+0000", 2024-09-15T01:44:08.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413786+0000", 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413787+0000", 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8800000000000002e-07 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413829+0000", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2109e-05 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413854+0000", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4790000000000002e-05 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.413869+0000", 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4832e-05 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.414021+0000", 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015239100000000001 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.414058+0000", 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6198999999999997e-05 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:08.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.415202+0000", 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001144204 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:57.415348+0000", 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014639000000000001 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:08.802 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:08.803 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:08.880 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:08.872+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:08.910 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:08.930 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:08.952 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:08.952 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:08.952 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:08.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:08.973 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:08.981 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:08.976+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:09.009 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:09.039 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:09.060 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.061 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:09.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:09.085 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:09.086 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:09.080+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:09.108 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:09.137 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:09.159 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:09.183 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:09.183 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:09.176+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:09.185 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:09.186 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:09.187 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.187 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:09.284 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:09.276+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:09.348 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.348 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:09.385 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:09.379+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:09.460 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.460 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:09.486 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:09.479+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:09.541 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:09.542 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:09.542 INFO:tasks.thrashosds.thrasher:Removing osd 11, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:09.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 11 2024-09-15T01:44:09.559 INFO:tasks.rados.rados.0.smithi033.stdout:1: write initial oid 1 2024-09-15T01:44:09.559 INFO:tasks.rados.rados.0.smithi033.stdout:1: seq_num 0 ranges {0=32768} 2024-09-15T01:44:09.559 INFO:tasks.rados.rados.0.smithi033.stdout:1: writing smithi03327189-1 from 0 to 32768 tid 1 2024-09-15T01:44:09.559 INFO:tasks.rados.rados.0.smithi033.stdout:2: write initial oid 2 2024-09-15T01:44:09.559 INFO:tasks.rados.rados.0.smithi033.stdout:2: seq_num 1 ranges {0=32768} 2024-09-15T01:44:09.560 INFO:tasks.rados.rados.0.smithi033.stdout:2: writing smithi03327189-2 from 0 to 32768 tid 1 2024-09-15T01:44:09.560 INFO:tasks.rados.rados.0.smithi033.stdout:3: write initial oid 3 2024-09-15T01:44:09.560 INFO:tasks.rados.rados.0.smithi033.stdout:3: seq_num 2 ranges {0=49152} 2024-09-15T01:44:09.560 INFO:tasks.rados.rados.0.smithi033.stdout:3: writing smithi03327189-3 from 0 to 49152 tid 1 2024-09-15T01:44:09.560 INFO:tasks.rados.rados.0.smithi033.stdout:4: write initial oid 4 2024-09-15T01:44:09.561 INFO:tasks.rados.rados.0.smithi033.stdout:4: seq_num 3 ranges {0=65536} 2024-09-15T01:44:09.565 INFO:tasks.rados.rados.0.smithi033.stdout:4: writing smithi03327189-4 from 0 to 65536 tid 1 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:5: write initial oid 5 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:5: seq_num 4 ranges {0=81920} 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:5: writing smithi03327189-5 from 0 to 81920 tid 1 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:6: write initial oid 6 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:6: seq_num 5 ranges {0=98304} 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:6: writing smithi03327189-6 from 0 to 98304 tid 1 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:7: write initial oid 7 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:7: seq_num 6 ranges {0=114688} 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:7: writing smithi03327189-7 from 0 to 114688 tid 1 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:8: write initial oid 8 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:8: seq_num 7 ranges {0=131072} 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:8: writing smithi03327189-8 from 0 to 131072 tid 1 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:9: write initial oid 9 2024-09-15T01:44:09.566 INFO:tasks.rados.rados.0.smithi033.stdout:9: seq_num 8 ranges {0=147456} 2024-09-15T01:44:09.567 INFO:tasks.rados.rados.0.smithi033.stdout:9: writing smithi03327189-9 from 0 to 147456 tid 1 2024-09-15T01:44:09.568 INFO:tasks.rados.rados.0.smithi033.stdout:10: write initial oid 10 2024-09-15T01:44:09.568 INFO:tasks.rados.rados.0.smithi033.stdout:10: seq_num 9 ranges {0=163840} 2024-09-15T01:44:09.571 INFO:tasks.rados.rados.0.smithi033.stdout:10: writing smithi03327189-10 from 0 to 163840 tid 1 2024-09-15T01:44:09.571 INFO:tasks.rados.rados.0.smithi033.stdout:11: write initial oid 11 2024-09-15T01:44:09.571 INFO:tasks.rados.rados.0.smithi033.stdout:11: seq_num 10 ranges {0=180224} 2024-09-15T01:44:09.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:44:09.582 INFO:tasks.rados.rados.0.smithi033.stdout:11: writing smithi03327189-11 from 0 to 180224 tid 1 2024-09-15T01:44:09.585 INFO:tasks.rados.rados.0.smithi033.stdout:2: finishing write tid 1 to smithi03327189-2 2024-09-15T01:44:09.585 INFO:tasks.rados.rados.0.smithi033.stdout:2: finishing write tid 2 to smithi03327189-2 2024-09-15T01:44:09.585 INFO:tasks.rados.rados.0.smithi033.stdout:2: finishing write tid 3 to smithi03327189-2 2024-09-15T01:44:09.585 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T01:44:09.585 INFO:tasks.rados.rados.0.smithi033.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:4: finishing write tid 1 to smithi03327189-4 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:4: finishing write tid 2 to smithi03327189-4 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:9: finishing write tid 1 to smithi03327189-9 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:1: finishing write tid 1 to smithi03327189-1 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:7: finishing write tid 1 to smithi03327189-7 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:7: finishing write tid 2 to smithi03327189-7 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:3: finishing write tid 1 to smithi03327189-3 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:3: finishing write tid 2 to smithi03327189-3 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:3: finishing write tid 3 to smithi03327189-3 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:7: finishing write tid 3 to smithi03327189-7 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-15T01:44:09.586 INFO:tasks.rados.rados.0.smithi033.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:6: finishing write tid 1 to smithi03327189-6 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:6: finishing write tid 2 to smithi03327189-6 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:9: finishing write tid 2 to smithi03327189-9 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:1: finishing write tid 2 to smithi03327189-1 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:10: finishing write tid 1 to smithi03327189-10 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:10: finishing write tid 2 to smithi03327189-10 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:8: finishing write tid 1 to smithi03327189-8 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:4: finishing write tid 3 to smithi03327189-4 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:2: done (10 left) 2024-09-15T01:44:09.587 INFO:tasks.rados.rados.0.smithi033.stdout:3: done (9 left) 2024-09-15T01:44:09.588 INFO:tasks.rados.rados.0.smithi033.stdout:4: done (8 left) 2024-09-15T01:44:09.588 INFO:tasks.rados.rados.0.smithi033.stdout:7: done (7 left) 2024-09-15T01:44:09.588 INFO:tasks.rados.rados.0.smithi033.stdout:12: write initial oid 12 2024-09-15T01:44:09.588 INFO:tasks.rados.rados.0.smithi033.stdout:12: seq_num 11 ranges {0=196608} 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:12: writing smithi03327189-12 from 0 to 196608 tid 1 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:8: finishing write tid 2 to smithi03327189-8 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:1: finishing write tid 3 to smithi03327189-1 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:9: finishing write tid 3 to smithi03327189-9 2024-09-15T01:44:09.589 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:5: finishing write tid 1 to smithi03327189-5 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:10: finishing write tid 3 to smithi03327189-10 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:1: done (7 left) 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:9: done (6 left) 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:10: done (5 left) 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:13: write initial oid 13 2024-09-15T01:44:09.590 INFO:tasks.rados.rados.0.smithi033.stdout:13: seq_num 12 ranges {0=212992} 2024-09-15T01:44:09.590 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:09.584+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:09.591 INFO:tasks.rados.rados.0.smithi033.stdout:13: writing smithi03327189-13 from 0 to 212992 tid 1 2024-09-15T01:44:09.591 INFO:tasks.rados.rados.0.smithi033.stdout:14: write initial oid 14 2024-09-15T01:44:09.591 INFO:tasks.rados.rados.0.smithi033.stdout:14: seq_num 13 ranges {0=229376} 2024-09-15T01:44:09.591 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.591 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.592 INFO:tasks.rados.rados.0.smithi033.stdout:14: writing smithi03327189-14 from 0 to 229376 tid 1 2024-09-15T01:44:09.592 INFO:tasks.rados.rados.0.smithi033.stdout:15: write initial oid 15 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:6: finishing write tid 3 to smithi03327189-6 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:8: finishing write tid 3 to smithi03327189-8 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:5: finishing write tid 2 to smithi03327189-5 2024-09-15T01:44:09.593 INFO:tasks.rados.rados.0.smithi033.stdout:15: seq_num 14 ranges {0=245760} 2024-09-15T01:44:09.595 INFO:tasks.rados.rados.0.smithi033.stdout:15: writing smithi03327189-15 from 0 to 245760 tid 1 2024-09-15T01:44:09.595 INFO:tasks.rados.rados.0.smithi033.stdout:6: done (7 left) 2024-09-15T01:44:09.595 INFO:tasks.rados.rados.0.smithi033.stdout:8: done (6 left) 2024-09-15T01:44:09.595 INFO:tasks.rados.rados.0.smithi033.stdout:16: write initial oid 16 2024-09-15T01:44:09.595 INFO:tasks.rados.rados.0.smithi033.stdout:16: seq_num 15 ranges {0=262144} 2024-09-15T01:44:09.598 INFO:tasks.rados.rados.0.smithi033.stdout:16: writing smithi03327189-16 from 0 to 262144 tid 1 2024-09-15T01:44:09.598 INFO:tasks.rados.rados.0.smithi033.stdout:17: write initial oid 17 2024-09-15T01:44:09.598 INFO:tasks.rados.rados.0.smithi033.stdout:17: seq_num 16 ranges {0=278528} 2024-09-15T01:44:09.601 INFO:tasks.rados.rados.0.smithi033.stdout:17: writing smithi03327189-17 from 0 to 278528 tid 1 2024-09-15T01:44:09.601 INFO:tasks.rados.rados.0.smithi033.stdout:18: write initial oid 18 2024-09-15T01:44:09.601 INFO:tasks.rados.rados.0.smithi033.stdout:18: seq_num 17 ranges {0=294912} 2024-09-15T01:44:09.605 INFO:tasks.rados.rados.0.smithi033.stdout:18: writing smithi03327189-18 from 0 to 294912 tid 1 2024-09-15T01:44:09.605 INFO:tasks.rados.rados.0.smithi033.stdout:19: write initial oid 19 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:11: finishing write tid 1 to smithi03327189-11 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:11: finishing write tid 2 to smithi03327189-11 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:5: finishing write tid 3 to smithi03327189-5 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:12: finishing write tid 1 to smithi03327189-12 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:13: finishing write tid 1 to smithi03327189-13 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:12: finishing write tid 2 to smithi03327189-12 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:11: finishing write tid 3 to smithi03327189-11 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:12: finishing write tid 3 to smithi03327189-12 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T01:44:09.606 INFO:tasks.rados.rados.0.smithi033.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:13: finishing write tid 2 to smithi03327189-13 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:15: finishing write tid 1 to smithi03327189-15 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:15: finishing write tid 2 to smithi03327189-15 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:14: finishing write tid 1 to smithi03327189-14 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:14: finishing write tid 2 to smithi03327189-14 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:13: finishing write tid 3 to smithi03327189-13 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-15T01:44:09.607 INFO:tasks.rados.rados.0.smithi033.stdout:19: seq_num 18 ranges {0=311296} 2024-09-15T01:44:09.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:19: writing smithi03327189-19 from 0 to 311296 tid 1 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:5: done (9 left) 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:11: done (8 left) 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:12: done (7 left) 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:13: done (6 left) 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:20: write initial oid 20 2024-09-15T01:44:09.609 INFO:tasks.rados.rados.0.smithi033.stdout:20: seq_num 19 ranges {0=327680} 2024-09-15T01:44:09.610 INFO:tasks.rados.rados.0.smithi033.stdout:20: writing smithi03327189-20 from 0 to 327680 tid 1 2024-09-15T01:44:09.610 INFO:tasks.rados.rados.0.smithi033.stdout:21: write initial oid 21 2024-09-15T01:44:09.610 INFO:tasks.rados.rados.0.smithi033.stdout:21: seq_num 20 ranges {0=344064} 2024-09-15T01:44:09.613 INFO:tasks.rados.rados.0.smithi033.stdout:21: writing smithi03327189-21 from 0 to 344064 tid 1 2024-09-15T01:44:09.613 INFO:tasks.rados.rados.0.smithi033.stdout:22: write initial oid 22 2024-09-15T01:44:09.613 INFO:tasks.rados.rados.0.smithi033.stdout:22: seq_num 21 ranges {0=360448} 2024-09-15T01:44:09.615 INFO:tasks.rados.rados.0.smithi033.stdout:22: writing smithi03327189-22 from 0 to 360448 tid 1 2024-09-15T01:44:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:23: write initial oid 23 2024-09-15T01:44:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:23: seq_num 22 ranges {0=376832} 2024-09-15T01:44:09.619 INFO:tasks.rados.rados.0.smithi033.stdout:23: writing smithi03327189-23 from 0 to 376832 tid 1 2024-09-15T01:44:09.619 INFO:tasks.rados.rados.0.smithi033.stdout:24: write initial oid 24 2024-09-15T01:44:09.619 INFO:tasks.rados.rados.0.smithi033.stdout:24: seq_num 23 ranges {0=393216} 2024-09-15T01:44:09.623 INFO:tasks.rados.rados.0.smithi033.stdout:24: writing smithi03327189-24 from 0 to 393216 tid 1 2024-09-15T01:44:09.623 INFO:tasks.rados.rados.0.smithi033.stdout:15: finishing write tid 3 to smithi03327189-15 2024-09-15T01:44:09.623 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:16: finishing write tid 1 to smithi03327189-16 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:14: finishing write tid 3 to smithi03327189-14 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:16: finishing write tid 2 to smithi03327189-16 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:16: finishing write tid 3 to smithi03327189-16 2024-09-15T01:44:09.624 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:17: finishing write tid 1 to smithi03327189-17 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:18: finishing write tid 1 to smithi03327189-18 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:17: finishing write tid 2 to smithi03327189-17 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:18: finishing write tid 2 to smithi03327189-18 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:17: finishing write tid 3 to smithi03327189-17 2024-09-15T01:44:09.625 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:19: finishing write tid 1 to smithi03327189-19 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:19: finishing write tid 2 to smithi03327189-19 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:21: finishing write tid 1 to smithi03327189-21 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:20: finishing write tid 1 to smithi03327189-20 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:20: finishing write tid 2 to smithi03327189-20 2024-09-15T01:44:09.626 INFO:tasks.rados.rados.0.smithi033.stdout:21: finishing write tid 2 to smithi03327189-21 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:18: finishing write tid 3 to smithi03327189-18 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:14: done (10 left) 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:15: done (9 left) 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:16: done (8 left) 2024-09-15T01:44:09.627 INFO:tasks.rados.rados.0.smithi033.stdout:17: done (7 left) 2024-09-15T01:44:09.628 INFO:tasks.rados.rados.0.smithi033.stdout:18: done (6 left) 2024-09-15T01:44:09.628 INFO:tasks.rados.rados.0.smithi033.stdout:25: write initial oid 25 2024-09-15T01:44:09.628 INFO:tasks.rados.rados.0.smithi033.stdout:25: seq_num 24 ranges {0=409600} 2024-09-15T01:44:09.629 INFO:tasks.rados.rados.0.smithi033.stdout:25: writing smithi03327189-25 from 0 to 409600 tid 1 2024-09-15T01:44:09.629 INFO:tasks.rados.rados.0.smithi033.stdout:26: write initial oid 26 2024-09-15T01:44:09.630 INFO:tasks.rados.rados.0.smithi033.stdout:26: seq_num 25 ranges {0=425984} 2024-09-15T01:44:09.630 INFO:tasks.rados.rados.0.smithi033.stdout:26: writing smithi03327189-26 from 0 to 425984 tid 1 2024-09-15T01:44:09.630 INFO:tasks.rados.rados.0.smithi033.stdout:27: write initial oid 27 2024-09-15T01:44:09.630 INFO:tasks.rados.rados.0.smithi033.stdout:27: seq_num 26 ranges {0=442368} 2024-09-15T01:44:09.634 INFO:tasks.rados.rados.0.smithi033.stdout:27: writing smithi03327189-27 from 0 to 442368 tid 1 2024-09-15T01:44:09.634 INFO:tasks.rados.rados.0.smithi033.stdout:28: write initial oid 28 2024-09-15T01:44:09.634 INFO:tasks.rados.rados.0.smithi033.stdout:28: seq_num 27 ranges {0=458752} 2024-09-15T01:44:09.638 INFO:tasks.rados.rados.0.smithi033.stdout:28: writing smithi03327189-28 from 0 to 458752 tid 1 2024-09-15T01:44:09.638 INFO:tasks.rados.rados.0.smithi033.stdout:29: write initial oid 29 2024-09-15T01:44:09.638 INFO:tasks.rados.rados.0.smithi033.stdout:29: seq_num 28 ranges {0=475136} 2024-09-15T01:44:09.641 INFO:tasks.rados.rados.0.smithi033.stdout:29: writing smithi03327189-29 from 0 to 475136 tid 1 2024-09-15T01:44:09.641 INFO:tasks.rados.rados.0.smithi033.stdout:30: write initial oid 30 2024-09-15T01:44:09.642 INFO:tasks.rados.rados.0.smithi033.stdout:30: seq_num 29 ranges {0=491520} 2024-09-15T01:44:09.645 INFO:tasks.rados.rados.0.smithi033.stdout:30: writing smithi03327189-30 from 0 to 491520 tid 1 2024-09-15T01:44:09.645 INFO:tasks.rados.rados.0.smithi033.stdout:31: write initial oid 31 2024-09-15T01:44:09.645 INFO:tasks.rados.rados.0.smithi033.stdout:31: seq_num 30 ranges {0=507904} 2024-09-15T01:44:09.648 INFO:tasks.rados.rados.0.smithi033.stdout:31: writing smithi03327189-31 from 0 to 507904 tid 1 2024-09-15T01:44:09.649 INFO:tasks.rados.rados.0.smithi033.stdout:32: write initial oid 32 2024-09-15T01:44:09.649 INFO:tasks.rados.rados.0.smithi033.stdout:32: seq_num 31 ranges {0=524288} 2024-09-15T01:44:09.653 INFO:tasks.rados.rados.0.smithi033.stdout:32: writing smithi03327189-32 from 0 to 524288 tid 1 2024-09-15T01:44:09.653 INFO:tasks.rados.rados.0.smithi033.stdout:33: write initial oid 33 2024-09-15T01:44:09.653 INFO:tasks.rados.rados.0.smithi033.stdout:33: seq_num 32 ranges {0=540672} 2024-09-15T01:44:09.657 INFO:tasks.rados.rados.0.smithi033.stdout:33: writing smithi03327189-33 from 0 to 540672 tid 1 2024-09-15T01:44:09.657 INFO:tasks.rados.rados.0.smithi033.stdout:34: write initial oid 34 2024-09-15T01:44:09.657 INFO:tasks.rados.rados.0.smithi033.stdout:34: seq_num 33 ranges {0=557056} 2024-09-15T01:44:09.661 INFO:tasks.rados.rados.0.smithi033.stdout:34: writing smithi03327189-34 from 0 to 557056 tid 1 2024-09-15T01:44:09.661 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.661 INFO:tasks.rados.rados.0.smithi033.stdout:19: finishing write tid 3 to smithi03327189-19 2024-09-15T01:44:09.661 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 19 v 2 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T01:44:09.661 INFO:tasks.rados.rados.0.smithi033.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:44:09.662 INFO:tasks.rados.rados.0.smithi033.stdout:19: done (15 left) 2024-09-15T01:44:09.662 INFO:tasks.rados.rados.0.smithi033.stdout:35: write initial oid 35 2024-09-15T01:44:09.662 INFO:tasks.rados.rados.0.smithi033.stdout:35: seq_num 34 ranges {0=573440} 2024-09-15T01:44:09.665 INFO:tasks.rados.rados.0.smithi033.stdout:35: writing smithi03327189-35 from 0 to 573440 tid 1 2024-09-15T01:44:09.665 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.665 INFO:tasks.rados.rados.0.smithi033.stdout:21: finishing write tid 3 to smithi03327189-21 2024-09-15T01:44:09.665 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 21 v 4 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T01:44:09.666 INFO:tasks.rados.rados.0.smithi033.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-15T01:44:09.666 INFO:tasks.rados.rados.0.smithi033.stdout:21: done (15 left) 2024-09-15T01:44:09.666 INFO:tasks.rados.rados.0.smithi033.stdout:36: write initial oid 36 2024-09-15T01:44:09.666 INFO:tasks.rados.rados.0.smithi033.stdout:36: seq_num 35 ranges {0=589824} 2024-09-15T01:44:09.671 INFO:tasks.rados.rados.0.smithi033.stdout:36: writing smithi03327189-36 from 0 to 589824 tid 1 2024-09-15T01:44:09.671 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.671 INFO:tasks.rados.rados.0.smithi033.stdout:22: finishing write tid 1 to smithi03327189-22 2024-09-15T01:44:09.671 INFO:tasks.rados.rados.0.smithi033.stdout:20: finishing write tid 3 to smithi03327189-20 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:22: finishing write tid 2 to smithi03327189-22 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:22: finishing write tid 3 to smithi03327189-22 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 22 v 6 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:23: finishing write tid 1 to smithi03327189-23 2024-09-15T01:44:09.672 INFO:tasks.rados.rados.0.smithi033.stdout:23: finishing write tid 2 to smithi03327189-23 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:23: finishing write tid 3 to smithi03327189-23 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 23 v 2 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:25: finishing write tid 1 to smithi03327189-25 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:24: finishing write tid 1 to smithi03327189-24 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:25: finishing write tid 2 to smithi03327189-25 2024-09-15T01:44:09.673 INFO:tasks.rados.rados.0.smithi033.stdout:24: finishing write tid 2 to smithi03327189-24 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:26: finishing write tid 1 to smithi03327189-26 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:24: finishing write tid 3 to smithi03327189-24 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:20: done (15 left) 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:22: done (14 left) 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:23: done (13 left) 2024-09-15T01:44:09.674 INFO:tasks.rados.rados.0.smithi033.stdout:24: done (12 left) 2024-09-15T01:44:09.675 INFO:tasks.rados.rados.0.smithi033.stdout:37: write initial oid 37 2024-09-15T01:44:09.675 INFO:tasks.rados.rados.0.smithi033.stdout:37: seq_num 36 ranges {0=606208} 2024-09-15T01:44:09.675 INFO:tasks.rados.rados.0.smithi033.stdout:37: writing smithi03327189-37 from 0 to 606208 tid 1 2024-09-15T01:44:09.675 INFO:tasks.rados.rados.0.smithi033.stdout:38: write initial oid 38 2024-09-15T01:44:09.676 INFO:tasks.rados.rados.0.smithi033.stdout:38: seq_num 37 ranges {0=622592} 2024-09-15T01:44:09.679 INFO:tasks.rados.rados.0.smithi033.stdout:38: writing smithi03327189-38 from 0 to 622592 tid 1 2024-09-15T01:44:09.679 INFO:tasks.rados.rados.0.smithi033.stdout:39: write initial oid 39 2024-09-15T01:44:09.679 INFO:tasks.rados.rados.0.smithi033.stdout:39: seq_num 38 ranges {0=638976} 2024-09-15T01:44:09.684 INFO:tasks.rados.rados.0.smithi033.stdout:39: writing smithi03327189-39 from 0 to 638976 tid 1 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:25: finishing write tid 3 to smithi03327189-25 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 25 v 2 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:26: finishing write tid 2 to smithi03327189-26 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:27: finishing write tid 1 to smithi03327189-27 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:25: done (14 left) 2024-09-15T01:44:09.685 INFO:tasks.rados.rados.0.smithi033.stdout:40: write initial oid 40 2024-09-15T01:44:09.686 INFO:tasks.rados.rados.0.smithi033.stdout:27: finishing write tid 2 to smithi03327189-27 2024-09-15T01:44:09.686 INFO:tasks.rados.rados.0.smithi033.stdout:40: seq_num 39 ranges {0=671744} 2024-09-15T01:44:09.689 INFO:tasks.rados.rados.0.smithi033.stdout:40: writing smithi03327189-40 from 0 to 671744 tid 1 2024-09-15T01:44:09.689 INFO:tasks.rados.rados.0.smithi033.stdout:41: write initial oid 41 2024-09-15T01:44:09.690 INFO:tasks.rados.rados.0.smithi033.stdout:41: seq_num 40 ranges {0=688128} 2024-09-15T01:44:09.690 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:09.684+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:41: writing smithi03327189-41 from 0 to 688128 tid 1 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:26: finishing write tid 3 to smithi03327189-26 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:26: done (15 left) 2024-09-15T01:44:09.695 INFO:tasks.rados.rados.0.smithi033.stdout:42: write initial oid 42 2024-09-15T01:44:09.696 INFO:tasks.rados.rados.0.smithi033.stdout:42: seq_num 41 ranges {0=704512} 2024-09-15T01:44:09.701 INFO:tasks.rados.rados.0.smithi033.stdout:42: writing smithi03327189-42 from 0 to 704512 tid 1 2024-09-15T01:44:09.701 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:27: finishing write tid 3 to smithi03327189-27 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:27: done (15 left) 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:43: write initial oid 43 2024-09-15T01:44:09.702 INFO:tasks.rados.rados.0.smithi033.stdout:43: seq_num 42 ranges {0=720896} 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:43: writing smithi03327189-43 from 0 to 720896 tid 1 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:28: finishing write tid 1 to smithi03327189-28 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:28: finishing write tid 2 to smithi03327189-28 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:29: finishing write tid 1 to smithi03327189-29 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:29: finishing write tid 2 to smithi03327189-29 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:28: finishing write tid 3 to smithi03327189-28 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 28 v 4 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:29: finishing write tid 3 to smithi03327189-29 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 29 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-15T01:44:09.706 INFO:tasks.rados.rados.0.smithi033.stdout:30: finishing write tid 1 to smithi03327189-30 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:30: finishing write tid 2 to smithi03327189-30 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:31: finishing write tid 1 to smithi03327189-31 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:31: finishing write tid 2 to smithi03327189-31 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:32: finishing write tid 1 to smithi03327189-32 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:31: finishing write tid 3 to smithi03327189-31 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 31 v 8 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:30: finishing write tid 3 to smithi03327189-30 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:32: finishing write tid 2 to smithi03327189-32 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:28: done (15 left) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:29: done (14 left) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:30: done (13 left) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:31: done (12 left) 2024-09-15T01:44:09.707 INFO:tasks.rados.rados.0.smithi033.stdout:44: write initial oid 44 2024-09-15T01:44:09.708 INFO:tasks.rados.rados.0.smithi033.stdout:44: seq_num 43 ranges {0=737280} 2024-09-15T01:44:09.710 INFO:tasks.rados.rados.0.smithi033.stdout:44: writing smithi03327189-44 from 0 to 737280 tid 1 2024-09-15T01:44:09.711 INFO:tasks.rados.rados.0.smithi033.stdout:45: write initial oid 45 2024-09-15T01:44:09.711 INFO:tasks.rados.rados.0.smithi033.stdout:45: seq_num 44 ranges {0=753664} 2024-09-15T01:44:09.716 INFO:tasks.rados.rados.0.smithi033.stdout:45: writing smithi03327189-45 from 0 to 753664 tid 1 2024-09-15T01:44:09.716 INFO:tasks.rados.rados.0.smithi033.stdout:46: write initial oid 46 2024-09-15T01:44:09.716 INFO:tasks.rados.rados.0.smithi033.stdout:46: seq_num 45 ranges {0=770048} 2024-09-15T01:44:09.722 INFO:tasks.rados.rados.0.smithi033.stdout:46: writing smithi03327189-46 from 0 to 770048 tid 1 2024-09-15T01:44:09.723 INFO:tasks.rados.rados.0.smithi033.stdout:47: write initial oid 47 2024-09-15T01:44:09.723 INFO:tasks.rados.rados.0.smithi033.stdout:47: seq_num 46 ranges {0=786432} 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:47: writing smithi03327189-47 from 0 to 786432 tid 1 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout: waiting on 16 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:32: finishing write tid 3 to smithi03327189-32 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 32 v 10 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:33: finishing write tid 1 to smithi03327189-33 2024-09-15T01:44:09.730 INFO:tasks.rados.rados.0.smithi033.stdout:33: finishing write tid 2 to smithi03327189-33 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:34: finishing write tid 1 to smithi03327189-34 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:33: finishing write tid 3 to smithi03327189-33 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 33 v 4 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:34: finishing write tid 2 to smithi03327189-34 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:35: finishing write tid 1 to smithi03327189-35 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:35: finishing write tid 2 to smithi03327189-35 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:34: finishing write tid 3 to smithi03327189-34 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 34 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:35: finishing write tid 3 to smithi03327189-35 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-15T01:44:09.731 INFO:tasks.rados.rados.0.smithi033.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:36: finishing write tid 1 to smithi03327189-36 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:36: finishing write tid 2 to smithi03327189-36 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:37: finishing write tid 1 to smithi03327189-37 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:37: finishing write tid 2 to smithi03327189-37 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:36: finishing write tid 3 to smithi03327189-36 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 36 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:37: finishing write tid 3 to smithi03327189-37 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 37 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:38: finishing write tid 1 to smithi03327189-38 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:38: finishing write tid 2 to smithi03327189-38 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:38: finishing write tid 3 to smithi03327189-38 2024-09-15T01:44:09.732 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 10 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:39: finishing write tid 1 to smithi03327189-39 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:39: finishing write tid 2 to smithi03327189-39 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:39: finishing write tid 3 to smithi03327189-39 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:40: finishing write tid 1 to smithi03327189-40 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:40: finishing write tid 2 to smithi03327189-40 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:40: finishing write tid 3 to smithi03327189-40 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:41: finishing write tid 1 to smithi03327189-41 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:41: finishing write tid 2 to smithi03327189-41 2024-09-15T01:44:09.733 INFO:tasks.rados.rados.0.smithi033.stdout:32: done (15 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:33: done (14 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:34: done (13 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:35: done (12 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:36: done (11 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:37: done (10 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:38: done (9 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:39: done (8 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:40: done (7 left) 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:48: write initial oid 48 2024-09-15T01:44:09.734 INFO:tasks.rados.rados.0.smithi033.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-15T01:44:09.735 INFO:tasks.rados.rados.0.smithi033.stdout:48: writing smithi03327189-48 from 0 to 425984 tid 1 2024-09-15T01:44:09.735 INFO:tasks.rados.rados.0.smithi033.stdout:48: writing smithi03327189-48 from 425984 to 802816 tid 2 2024-09-15T01:44:09.735 INFO:tasks.rados.rados.0.smithi033.stdout:49: write initial oid 49 2024-09-15T01:44:09.735 INFO:tasks.rados.rados.0.smithi033.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-15T01:44:09.738 INFO:tasks.rados.rados.0.smithi033.stdout:49: writing smithi03327189-49 from 0 to 442368 tid 1 2024-09-15T01:44:09.740 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.740 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.741 INFO:tasks.rados.rados.0.smithi033.stdout:49: writing smithi03327189-49 from 442368 to 819200 tid 2 2024-09-15T01:44:09.741 INFO:tasks.rados.rados.0.smithi033.stdout:50: write initial oid 50 2024-09-15T01:44:09.742 INFO:tasks.rados.rados.0.smithi033.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-15T01:44:09.745 INFO:tasks.rados.rados.0.smithi033.stdout:50: writing smithi03327189-50 from 0 to 458752 tid 1 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:50: writing smithi03327189-50 from 458752 to 835584 tid 2 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:51: delete oid 8 current snap is 0 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:41: finishing write tid 3 to smithi03327189-41 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 41 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:42: finishing write tid 1 to smithi03327189-42 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:42: finishing write tid 2 to smithi03327189-42 2024-09-15T01:44:09.749 INFO:tasks.rados.rados.0.smithi033.stdout:42: finishing write tid 3 to smithi03327189-42 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:43: finishing write tid 1 to smithi03327189-43 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:43: finishing write tid 2 to smithi03327189-43 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:43: finishing write tid 3 to smithi03327189-43 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 43 v 4 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:44: finishing write tid 1 to smithi03327189-44 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:44: finishing write tid 2 to smithi03327189-44 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:44: finishing write tid 3 to smithi03327189-44 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-15T01:44:09.750 INFO:tasks.rados.rados.0.smithi033.stdout:45: finishing write tid 1 to smithi03327189-45 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:45: finishing write tid 2 to smithi03327189-45 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:45: finishing write tid 3 to smithi03327189-45 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 45 v 12 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:46: finishing write tid 1 to smithi03327189-46 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:46: finishing write tid 2 to smithi03327189-46 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:46: finishing write tid 3 to smithi03327189-46 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 46 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:47: finishing write tid 1 to smithi03327189-47 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:47: finishing write tid 2 to smithi03327189-47 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:47: finishing write tid 3 to smithi03327189-47 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 14 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T01:44:09.751 INFO:tasks.rados.rados.0.smithi033.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:48: finishing write tid 1 to smithi03327189-48 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:48: finishing write tid 2 to smithi03327189-48 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:48: finishing write tid 3 to smithi03327189-48 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:48: finishing write tid 4 to smithi03327189-48 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-15T01:44:09.752 INFO:tasks.rados.rados.0.smithi033.stdout:49: finishing write tid 1 to smithi03327189-49 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:49: finishing write tid 2 to smithi03327189-49 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:49: finishing write tid 3 to smithi03327189-49 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:50: finishing write tid 1 to smithi03327189-50 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:49: finishing write tid 4 to smithi03327189-49 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 49 v 5 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T01:44:09.754 INFO:tasks.rados.rados.0.smithi033.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-15T01:44:09.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:09.755 INFO:tasks.rados.rados.0.smithi033.stdout:41: done (10 left) 2024-09-15T01:44:09.755 INFO:tasks.rados.rados.0.smithi033.stdout:42: done (9 left) 2024-09-15T01:44:09.755 INFO:tasks.rados.rados.0.smithi033.stdout:43: done (8 left) 2024-09-15T01:44:09.755 INFO:tasks.rados.rados.0.smithi033.stdout:44: done (7 left) 2024-09-15T01:44:09.755 INFO:tasks.rados.rados.0.smithi033.stdout:45: done (6 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:46: done (5 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:47: done (4 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:48: done (3 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:49: done (2 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:51: done (1 left) 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 31 current snap is 0 2024-09-15T01:44:09.756 INFO:tasks.rados.rados.0.smithi033.stdout:52: seq_num 51 ranges {507904=491520,999424=376832} 2024-09-15T01:44:09.759 INFO:tasks.rados.rados.0.smithi033.stdout:52: writing smithi03327189-31 from 507904 to 999424 tid 1 2024-09-15T01:44:09.762 INFO:tasks.rados.rados.0.smithi033.stdout:52: writing smithi03327189-31 from 999424 to 1376256 tid 2 2024-09-15T01:44:09.762 INFO:tasks.rados.rados.0.smithi033.stdout:53: rollback oid 46 current snap is 0 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:53: done (2 left) 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:54: setattr oid 25 current snap is 0 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:50: finishing write tid 2 to smithi03327189-50 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:50: finishing write tid 3 to smithi03327189-50 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:50: finishing write tid 4 to smithi03327189-50 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 50 v 17 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T01:44:09.763 INFO:tasks.rados.rados.0.smithi033.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-15T01:44:09.764 INFO:tasks.rados.rados.0.smithi033.stdout:50: done (2 left) 2024-09-15T01:44:09.764 INFO:tasks.rados.rados.0.smithi033.stdout:55: rollback oid 38 current snap is 0 2024-09-15T01:44:09.764 INFO:tasks.rados.rados.0.smithi033.stdout:55: done (2 left) 2024-09-15T01:44:09.764 INFO:tasks.rados.rados.0.smithi033.stdout:56: rollback oid 11 current snap is 0 2024-09-15T01:44:09.765 INFO:tasks.rados.rados.0.smithi033.stdout:56: done (2 left) 2024-09-15T01:44:09.765 INFO:tasks.rados.rados.0.smithi033.stdout:57: rmattr oid 24 current snap is 0 2024-09-15T01:44:09.765 INFO:tasks.rados.rados.0.smithi033.stdout:57: done (2 left) 2024-09-15T01:44:09.765 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 22 current snap is 0 2024-09-15T01:44:09.765 INFO:tasks.rados.rados.0.smithi033.stdout:58: seq_num 52 ranges {360448=507904,868352=376832} 2024-09-15T01:44:09.770 INFO:tasks.rados.rados.0.smithi033.stdout:58: writing smithi03327189-22 from 360448 to 868352 tid 1 2024-09-15T01:44:09.773 INFO:tasks.rados.rados.0.smithi033.stdout:58: writing smithi03327189-22 from 868352 to 1245184 tid 2 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 36 current snap is 0 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:52: finishing write tid 1 to smithi03327189-31 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:52: finishing write tid 2 to smithi03327189-31 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:52: finishing write tid 3 to smithi03327189-31 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 25 v 5 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:44:09.774 INFO:tasks.rados.rados.0.smithi033.stdout:59: seq_num 53 ranges {589824=524288,1114112=376832} 2024-09-15T01:44:09.782 INFO:tasks.rados.rados.0.smithi033.stdout:59: writing smithi03327189-36 from 589824 to 1114112 tid 1 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:59: writing smithi03327189-36 from 1114112 to 1490944 tid 2 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:54: done (3 left) 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:60: read oid 42 snap -1 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:60: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:61: read oid 2 snap -1 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:61: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:62: copy_from oid 41 from oid 5 current snap is 0 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:63: rollback oid 34 current snap is 0 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:63: done (6 left) 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:64: read oid 6 snap -1 2024-09-15T01:44:09.788 INFO:tasks.rados.rados.0.smithi033.stdout:64: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:65: read oid 19 snap -1 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:65: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:66: setattr oid 24 current snap is 0 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:52: finishing write tid 4 to smithi03327189-31 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 31 v 20 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:52: left oid 31 (ObjNum 51 snap 0 seq_num 51) 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:58: finishing write tid 1 to smithi03327189-22 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:58: finishing write tid 2 to smithi03327189-22 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:58: finishing write tid 3 to smithi03327189-22 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:58: finishing write tid 4 to smithi03327189-22 2024-09-15T01:44:09.789 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 22 v 9 (ObjNum 52 snap 0 seq_num 52) dirty exists 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:58: left oid 22 (ObjNum 52 snap 0 seq_num 52) 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:52: done (8 left) 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:58: done (7 left) 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:67: read oid 7 snap -1 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:67: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 46 current snap is 0 2024-09-15T01:44:09.790 INFO:tasks.rados.rados.0.smithi033.stdout:68: seq_num 54 ranges {770048=540672,1310720=376832} 2024-09-15T01:44:09.792 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:09.783+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:09.796 INFO:tasks.rados.rados.0.smithi033.stdout:68: writing smithi03327189-46 from 770048 to 1310720 tid 1 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:68: writing smithi03327189-46 from 1310720 to 1687552 tid 2 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:69: delete oid 17 current snap is 0 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:59: finishing write tid 1 to smithi03327189-36 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:59: finishing write tid 2 to smithi03327189-36 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:59: finishing write tid 3 to smithi03327189-36 2024-09-15T01:44:09.801 INFO:tasks.rados.rados.0.smithi033.stdout:64: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:44:09.806 INFO:tasks.rados.rados.0.smithi033.stdout:67: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T01:44:09.809 INFO:tasks.rados.rados.0.smithi033.stdout:60: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:44:09.828 INFO:tasks.rados.rados.0.smithi033.stdout:61: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-15T01:44:09.829 INFO:tasks.rados.rados.0.smithi033.stdout:65: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:62: finishing copy_from racing read to smithi03327189-41 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:59: finishing write tid 4 to smithi03327189-36 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 36 v 8 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:59: left oid 36 (ObjNum 53 snap 0 seq_num 53) 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:62: finishing copy_from to smithi03327189-41 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 41 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:59: done (9 left) 2024-09-15T01:44:09.838 INFO:tasks.rados.rados.0.smithi033.stdout:60: done (8 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:61: done (7 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:62: done (6 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:64: done (5 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:65: done (4 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:66: done (3 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:67: done (2 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:69: done (1 left) 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:70: delete oid 48 current snap is 0 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:68: finishing write tid 1 to smithi03327189-46 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:68: finishing write tid 2 to smithi03327189-46 2024-09-15T01:44:09.839 INFO:tasks.rados.rados.0.smithi033.stdout:68: finishing write tid 3 to smithi03327189-46 2024-09-15T01:44:09.840 INFO:tasks.rados.rados.0.smithi033.stdout:68: finishing write tid 4 to smithi03327189-46 2024-09-15T01:44:09.840 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 46 v 12 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T01:44:09.840 INFO:tasks.rados.rados.0.smithi033.stdout:68: left oid 46 (ObjNum 54 snap 0 seq_num 54) 2024-09-15T01:44:09.841 INFO:tasks.rados.rados.0.smithi033.stdout:68: done (1 left) 2024-09-15T01:44:09.841 INFO:tasks.rados.rados.0.smithi033.stdout:70: done (0 left) 2024-09-15T01:44:09.841 INFO:tasks.rados.rados.0.smithi033.stdout:71: delete oid 20 current snap is 0 2024-09-15T01:44:09.846 INFO:tasks.rados.rados.0.smithi033.stdout:71: done (0 left) 2024-09-15T01:44:09.846 INFO:tasks.rados.rados.0.smithi033.stdout:72: snap_create 2024-09-15T01:44:09.859 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.859 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:09.892 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:09.883+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:09.926 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-09-15T01:44:09.920+0000 7fed2b4c9700 -1 mon.a@0(leader).osd e30 definitely_dead 0 2024-09-15T01:44:09.965 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:09.965 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:09.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:09.993 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:09.987+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:10.076 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.076 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:10.097 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:10.087+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:10.182 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.182 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:10.196 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:10.188+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:10.288 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.289 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.297 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:10.288+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:10.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:10.396 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.396 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.397 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:10.392+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:10.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:10.497 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:10.492+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:10.507 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.507 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:72: done (0 left) 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:73: read oid 48 snap 0 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:73: expect deleted 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:74: copy_from oid 35 from oid 37 current snap is 1 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:75: rollback oid 5 current snap is 1 2024-09-15T01:44:10.575 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 5 to 0 2024-09-15T01:44:10.576 INFO:tasks.rados.rados.0.smithi033.stdout:76: snap_create 2024-09-15T01:44:10.593 INFO:teuthology.orchestra.run.smithi033.stderr:marked out osd.11. 2024-09-15T01:44:10.594 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:44:10.598 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:10.591+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:10.626 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.626 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:10.700 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:10.692+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:10.731 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.731 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:10.801 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:10.795+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:10.843 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.843 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:10.902 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:10.899+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:10.949 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:10.949 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:10.958 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:11.005 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:10.999+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:11.067 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:11.067 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:11.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:11.103 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:11.096+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:11.177 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:11.177 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:11.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:11.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:11.205 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:11.200+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:11.283 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:11.283 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:11.291 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:11.336 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:11.331+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:11.350 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:11.373 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:11.393 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:11.393 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:11.394 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:11.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:11.405 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:11.400+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:11.416 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:11.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:11.463 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:11.488 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:11.500 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:11.500 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:11.509 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:11.500+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:11.509 INFO:tasks.rados.rados.0.smithi033.stdout:76: done (3 left) 2024-09-15T01:44:11.509 INFO:tasks.rados.rados.0.smithi033.stdout:77: read oid 33 snap 0 2024-09-15T01:44:11.510 INFO:tasks.rados.rados.0.smithi033.stdout:77: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:44:11.510 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 20 current snap is 2 2024-09-15T01:44:11.510 INFO:tasks.rados.rados.0.smithi033.stdout:78: seq_num 58 ranges {0=606208,606208=376832} 2024-09-15T01:44:11.513 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:11.519 INFO:tasks.rados.rados.0.smithi033.stdout:78: writing smithi03327189-20 from 0 to 606208 tid 1 2024-09-15T01:44:11.520 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:11.522 INFO:tasks.rados.rados.0.smithi033.stdout:78: writing smithi03327189-20 from 606208 to 983040 tid 2 2024-09-15T01:44:11.523 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 14 current snap is 2 2024-09-15T01:44:11.523 INFO:tasks.rados.rados.0.smithi033.stdout:79: seq_num 59 ranges {229376=622592,851968=376832} 2024-09-15T01:44:11.528 INFO:tasks.rados.rados.0.smithi033.stdout:79: writing smithi03327189-14 from 229376 to 851968 tid 1 2024-09-15T01:44:11.530 INFO:tasks.rados.rados.0.smithi033.stdout:79: writing smithi03327189-14 from 851968 to 1228800 tid 2 2024-09-15T01:44:11.531 INFO:tasks.rados.rados.0.smithi033.stdout:80: snap_remove snap 1 2024-09-15T01:44:11.542 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:11.561 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:11.586 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:11.607 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:11.607 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:11.607 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:11.607 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [] 2024-09-15T01:44:11.607 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:11.609 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:11.604+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:11.616 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:11.708 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:11.704+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:11.714 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:11.714 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:11.714 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:11.714 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:11.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:11.809 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:11.804+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:11.823 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:11.823 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:11.823 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:11.823 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:11.823 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:11.832 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:11.909 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:11.904+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:11.931 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [] 2024-09-15T01:44:11.932 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:11.941 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:12.010 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:12.004+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:12.043 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:12.043 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:12.043 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:12.043 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:12.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:12.111 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:12.103+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:12.154 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:12.155 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:12.155 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:12.155 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:12.155 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:12.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:12.211 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:12.204+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.007518997999998, 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027436790000000002, 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.261 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248573+0000", 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.262 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248574+0000", 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.327e-06 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248717+0000", 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014298300000000001 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.263 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248751+0000", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3615999999999998e-05 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248761+0000", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0346000000000001e-05 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248855+0000", 2024-09-15T01:44:12.264 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3807999999999994e-05 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.248890+0000", 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.5099999999999999e-05 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.251228+0000", 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0023381169999999998 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.265 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.251317+0000", 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8381999999999995e-05 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.814110701000001, 2024-09-15T01:44:12.266 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.267 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:12.268 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.269 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:12.270 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:12.271 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.813149173999999, 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.810583934, 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.277 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.809622440999998, 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.807539882, 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.286 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.287 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:12.288 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.289 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:12.290 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.806508545, 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:12.291 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.292 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:12.293 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.294 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.805577996, 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.295 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.296 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:12.297 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:12.298 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.299 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.804722588000001, 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.300 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:12.301 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:12.302 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.303 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:50 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'32, mlcod=20'25)", 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.803796338000001, 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00272172, 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.304 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452296+0000", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452297+0000", 2024-09-15T01:44:12.305 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5030000000000001e-06 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452465+0000", 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00016775000000000001 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452496+0000", 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1312000000000001e-05 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.306 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452508+0000", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1873e-05 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452580+0000", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2071999999999997e-05 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.452624+0000", 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3936000000000001e-05 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.307 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454942+0000", 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0023181880000000001 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.455017+0000", 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.5086000000000006e-05 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.308 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.802206167000001, 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.309 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:12.310 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.311 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.250166454999999, 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.313 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.314 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:12.315 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.316 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.317 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.248752231000001, 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.318 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.319 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:12.320 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.321 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.247746208999999, 2024-09-15T01:44:12.322 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.323 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:12.324 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.325 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.326 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.246475580999999, 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.327 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.328 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:12.329 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:12.330 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:12.331 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:76 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=20'48)", 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:12.332 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 17.245499765000002, 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029957009999999999, 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.333 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010592+0000", 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010593+0000", 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010606+0000", 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3315999999999999e-05 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.334 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010621+0000", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4355000000000001e-05 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010626+0000", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6840000000000001e-06 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012455+0000", 2024-09-15T01:44:12.335 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0018292650000000001 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012507+0000", 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1099e-05 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013530+0000", 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00102333 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.336 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013588+0000", 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7886000000000001e-05 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:99 1.0 e21/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 21'73, mlcod=21'68)", 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 16.697588835000001, 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029651880000000001, 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.337 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558503+0000", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.558504+0000", 2024-09-15T01:44:12.338 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6499999999999998e-07 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559396+0000", 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00089181999999999998 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559422+0000", 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6482000000000002e-05 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.339 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559435+0000", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3107e-05 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559522+0000", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6501000000000004e-05 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.559562+0000", 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0209000000000001e-05 2024-09-15T01:44:12.340 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.561405+0000", 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001842865 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.561468+0000", 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3338999999999998e-05 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.341 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:101 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'75, mlcod=21'73)", 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 16.691666109, 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027461389999999999, 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:12.342 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564426+0000", 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.564427+0000", 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3370000000000001e-06 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.343 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565185+0000", 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00075745100000000002 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565212+0000", 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7042e-05 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.344 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565227+0000", 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5497000000000001e-05 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565350+0000", 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00012340300000000001 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565397+0000", 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.6875000000000001e-05 2024-09-15T01:44:12.345 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567111+0000", 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001713715 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567172+0000", 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.0819000000000002e-05 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.346 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 16.690777500999999, 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.347 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.348 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:12.349 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:12.350 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.351 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:103 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'77, mlcod=21'73)", 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 16.689788837999998, 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.002916709, 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.352 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566303+0000", 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566304+0000", 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.1399999999999997e-07 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567195+0000", 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00089143999999999996 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567219+0000", 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.4202999999999998e-05 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567246+0000", 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6356e-05 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567359+0000", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00011372800000000001 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.567407+0000", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.7741999999999998e-05 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569144+0000", 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001736958 2024-09-15T01:44:12.355 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569220+0000", 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.5667999999999994e-05 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:12.356 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:12.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:12.358 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:12.352+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:12.458 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:12.452+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3.2782bda1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.93852244299999998, 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.93856584399999998, 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3.2782bda1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.93609104499999995, 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.93624742100000002, 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3.2782bda1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.93585222999999995, 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.93610289000000002, 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3.2782bda1 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.93582084700000001, 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.93616456100000001, 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 4 2024-09-15T01:44:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:12.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:12.510 INFO:tasks.rados.rados.0.smithi033.stdout:80: done (6 left) 2024-09-15T01:44:12.510 INFO:tasks.rados.rados.0.smithi033.stdout:81: read oid 1 snap -1 2024-09-15T01:44:12.510 INFO:tasks.rados.rados.0.smithi033.stdout:81: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:44:12.510 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 15 current snap is 2 2024-09-15T01:44:12.510 INFO:tasks.rados.rados.0.smithi033.stdout:82: seq_num 60 ranges {245760=638976,884736=376832} 2024-09-15T01:44:12.515 INFO:tasks.rados.rados.0.smithi033.stdout:82: writing smithi03327189-15 from 245760 to 884736 tid 1 2024-09-15T01:44:12.518 INFO:tasks.rados.rados.0.smithi033.stdout:82: writing smithi03327189-15 from 884736 to 1261568 tid 2 2024-09-15T01:44:12.519 INFO:tasks.rados.rados.0.smithi033.stdout:83: read oid 47 snap -1 2024-09-15T01:44:12.519 INFO:tasks.rados.rados.0.smithi033.stdout:83: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T01:44:12.519 INFO:tasks.rados.rados.0.smithi033.stdout:84: read oid 28 snap -1 2024-09-15T01:44:12.519 INFO:tasks.rados.rados.0.smithi033.stdout:84: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-15T01:44:12.520 INFO:tasks.rados.rados.0.smithi033.stdout:85: read oid 19 snap 0 2024-09-15T01:44:12.520 INFO:tasks.rados.rados.0.smithi033.stdout:85: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:44:12.520 INFO:tasks.rados.rados.0.smithi033.stdout:86: snap_create 2024-09-15T01:44:12.520 INFO:tasks.rados.rados.0.smithi033.stdout:77: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:44:12.560 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:12.552+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:78: finishing write tid 1 to smithi03327189-20 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:78: finishing write tid 2 to smithi03327189-20 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:78: finishing write tid 3 to smithi03327189-20 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:78: finishing write tid 4 to smithi03327189-20 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 8 (ObjNum 58 snap 2 seq_num 58) dirty exists 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:78: left oid 20 (ObjNum 58 snap 2 seq_num 58) 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:82: finishing write tid 1 to smithi03327189-15 2024-09-15T01:44:12.563 INFO:tasks.rados.rados.0.smithi033.stdout:75: finishing rollback tid 0 to smithi03327189-5 2024-09-15T01:44:12.564 INFO:tasks.rados.rados.0.smithi033.stdout:85: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-15T01:44:12.575 INFO:tasks.rados.rados.0.smithi033.stdout:82: finishing write tid 2 to smithi03327189-15 2024-09-15T01:44:12.575 INFO:tasks.rados.rados.0.smithi033.stdout:84: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-15T01:44:12.591 INFO:tasks.rados.rados.0.smithi033.stdout:82: finishing write tid 3 to smithi03327189-15 2024-09-15T01:44:12.591 INFO:tasks.rados.rados.0.smithi033.stdout:82: finishing write tid 4 to smithi03327189-15 2024-09-15T01:44:12.591 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 6 (ObjNum 60 snap 2 seq_num 60) dirty exists 2024-09-15T01:44:12.591 INFO:tasks.rados.rados.0.smithi033.stdout:82: left oid 15 (ObjNum 60 snap 2 seq_num 60) 2024-09-15T01:44:12.591 INFO:tasks.rados.rados.0.smithi033.stdout:83: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T01:44:12.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:44:12.614 INFO:tasks.rados.rados.0.smithi033.stdout:74: finishing copy_from racing read to smithi03327189-35 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:79: finishing write tid 1 to smithi03327189-14 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:74: finishing copy_from to smithi03327189-35 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:79: finishing write tid 2 to smithi03327189-14 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:79: finishing write tid 3 to smithi03327189-14 2024-09-15T01:44:12.615 INFO:tasks.rados.rados.0.smithi033.stdout:81: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:79: finishing write tid 4 to smithi03327189-14 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 10 (ObjNum 59 snap 2 seq_num 59) dirty exists 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:79: left oid 14 (ObjNum 59 snap 2 seq_num 59) 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:75: finishing rollback tid 1 to smithi03327189-5 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:75: finishing rollback tid 2 to smithi03327189-5 2024-09-15T01:44:12.616 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 5 v 24 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T01:44:12.619 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:12.619 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:12.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:12.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:12.672+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:12.761 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:12.756+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:12.779 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2496385160000001, 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2472071179999999, 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2469683030000001, 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.24693692, 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.26772949000000001, 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.25981508800000003, 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.25978582500000003, 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.23234897900000001, 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.23118223399999999, 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.23080896100000001, 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.23008986300000001, 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.22860075699999999, 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00703289, 2024-09-15T01:44:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547486+0000", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547490+0000", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1160000000000001e-06 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547502+0000", 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2284e-05 2024-09-15T01:44:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547509+0000", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8709999999999999e-06 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554232+0000", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067227739999999999 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554518+0000", 2024-09-15T01:44:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028611400000000002 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.22853747399999999, 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006755016, 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547552+0000", 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.875e-06 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547560+0000", 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2099999999999993e-06 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547565+0000", 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.843e-06 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554285+0000", 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067204109999999999 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554304+0000", 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8223000000000002e-05 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.22850016100000001, 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074053929999999997, 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547589+0000", 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7970000000000001e-06 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547597+0000", 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7959999999999996e-06 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547601+0000", 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4259999999999996e-06 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554329+0000", 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067279790000000003 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554991+0000", 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000661928 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=30, last_complete=33'9, committed=1, applied=1))", 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.227577371, 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070855279999999998, 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548510+0000", 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3689999999999999e-06 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548515+0000", 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.259e-06 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548532+0000", 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6586999999999999e-05 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548540+0000", 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7670000000000003e-06 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555036+0000", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006496155 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555594+0000", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00055839099999999999 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=30, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.22662126699999999, 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061816969999999999, 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549466+0000", 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549471+0000", 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.344e-06 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549483+0000", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1627000000000001e-05 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9030000000000001e-06 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555629+0000", 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061406680000000002 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555646+0000", 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7325000000000001e-05 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.21998662199999999, 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067647840000000002, 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556100+0000", 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4499999999999998e-07 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556180+0000", 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9863000000000004e-05 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556188+0000", 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556193+0000", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7709999999999997e-06 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562617+0000", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064236800000000002 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562864+0000", 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000247366 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.21995521900000001, 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064187080000000004, 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556155+0000", 2024-09-15T01:44:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4170999999999999e-05 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556162+0000", 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0190000000000001e-06 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561747+0000", 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055845870000000002 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562549+0000", 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000802931 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.21945268300000001, 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066845059999999998, 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556653+0000", 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0038999999999999e-05 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556659+0000", 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9510000000000003e-06 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562747+0000", 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060879640000000004 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563318+0000", 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00057055199999999999 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.21921048400000001, 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007571698, 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556895+0000", 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9117e-05 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556900+0000", 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2440000000000001e-06 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563373+0000", 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064735360000000002 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564447+0000", 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010738010000000001 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:12.870 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:12.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:12.872 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:12.864+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:12.971 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:12.971 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:12.972 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:12.968+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:12.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:13.074 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:13.067+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:13.082 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:13.082 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:13.092 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:13.174 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:13.167+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.2s5 32/27 ECSubRead(tid=36, to_read={3:4ddcdeb3:::smithi03327189-33:head=0,135168,0}, subchunks={3:4ddcdeb3:::smithi03327189-33:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.515817+0000", 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.6718853660000002, 2024-09-15T01:44:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024874670000000002, 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515817+0000", 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515817+0000", 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515817+0000", 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2699999999999999e-07 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515828+0000", 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0669999999999999e-05 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515840+0000", 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2561000000000001e-05 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515849+0000", 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3669999999999997e-06 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.516078+0000", 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022936699999999999 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.518304+0000", 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022259749999999998 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.2s5 32/27 ECSubRead(tid=37, to_read={3:4ddcdeb3:::smithi03327189-33:head=0,135168,0}, subchunks={3:4ddcdeb3:::smithi03327189-33:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.515865+0000", 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.671837094, 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026570690000000002, 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515865+0000", 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515865+0000", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515865+0000", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3599999999999999e-07 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515871+0000", 2024-09-15T01:44:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.857e-06 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515880+0000", 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1640000000000007e-06 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.515887+0000", 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5270000000000004e-06 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.518343+0000", 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024565289999999998 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.518522+0000", 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017855599999999999 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.2s5 32/27 ECSubWrite(tid=39, reqid=client.4632.0:206, at_version=32'6, trim_to=0'0, roll_forward_to=30'5))", 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.522424+0000", 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.6652783100000002, 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002828377, 2024-09-15T01:44:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522424+0000", 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522424+0000", 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522424+0000", 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:44:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522734+0000", 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030935799999999997 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522748+0000", 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4450999999999999e-05 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522773+0000", 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5086e-05 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522894+0000", 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121062 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522909+0000", 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4807000000000001e-05 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525215+0000", 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023060279999999999 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525252+0000", 2024-09-15T01:44:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6687000000000003e-05 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.2s5 32/27 ECSubWrite(tid=42, reqid=client.4632.0:208, at_version=0'0, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.527515+0000", 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.6601866090000001, 2024-09-15T01:44:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020123369999999999, 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527515+0000", 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527515+0000", 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527516+0000", 2024-09-15T01:44:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9699999999999995e-07 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527526+0000", 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6569999999999992e-06 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527538+0000", 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1839e-05 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527543+0000", 2024-09-15T01:44:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0880000000000002e-06 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527609+0000", 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5989999999999997e-05 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527622+0000", 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3067999999999999e-05 2024-09-15T01:44:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529498+0000", 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018766259999999999 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529528+0000", 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9272e-05 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.66671582699999998, 2024-09-15T01:44:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.666676822, 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.66664587799999997, 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.66119987199999997, 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004275114, 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526503+0000", 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2900000000000002e-07 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526853+0000", 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00035008499999999998 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526864+0000", 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1246e-05 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526871+0000", 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8299999999999998e-06 2024-09-15T01:44:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526941+0000", 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0291999999999997e-05 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526954+0000", 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2208000000000001e-05 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530752+0000", 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037984220000000001 2024-09-15T01:44:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530777+0000", 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5102000000000001e-05 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526902+0000", 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.66079944300000004, 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037794819999999998, 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526902+0000", 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526902+0000", 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526903+0000", 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8299999999999996e-07 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526931+0000", 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7285e-05 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526941+0000", 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0868e-05 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526954+0000", 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2486999999999999e-05 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527272+0000", 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031792300000000001 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527282+0000", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7820000000000005e-06 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530659+0000", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003376974 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530682+0000", 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.338e-05 2024-09-15T01:44:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.8s1 33/32 ECSubReadReply(tid=24, attrs_read=0))", 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.528884+0000", 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65881820999999996, 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036817, 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.528884+0000", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.528884+0000", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.528884+0000", 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5400000000000001e-07 2024-09-15T01:44:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529972+0000", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001087925 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529983+0000", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0513000000000001e-05 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529988+0000", 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6380000000000001e-06 2024-09-15T01:44:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530773+0000", 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00078435099999999997 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532565+0000", 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017926190000000001 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.8s1 33/32 ECSubReadReply(tid=25, attrs_read=0))", 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.530156+0000", 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65754579700000004, 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025276589999999998, 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530156+0000", 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530156+0000", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530157+0000", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3300000000000002e-07 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530228+0000", 2024-09-15T01:44:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0925000000000003e-05 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530237+0000", 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6469999999999993e-06 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530243+0000", 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2730000000000002e-06 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532611+0000", 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023679730000000002 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532684+0000", 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3207999999999994e-05 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.8s1 33/32 ECSubReadReply(tid=26, attrs_read=0))", 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.530463+0000", 2024-09-15T01:44:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.657238655, 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023168170000000001, 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530463+0000", 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530463+0000", 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530464+0000", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8599999999999998e-07 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530604+0000", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140533 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530613+0000", 2024-09-15T01:44:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8019999999999998e-06 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530618+0000", 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8359999999999999e-06 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532710+0000", 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020918450000000002 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532780+0000", 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0115000000000002e-05 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65573599500000002, 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0057431050000000001, 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531967+0000", 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000233353 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2055999999999999e-05 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.635e-06 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532530+0000", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031128500000000001 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532541+0000", 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0937999999999999e-05 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537675+0000", 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051336020000000001 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537709+0000", 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4424000000000001e-05 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=62, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.653259217, 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055656810000000003, 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534451+0000", 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5809999999999999e-06 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534467+0000", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6405e-05 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534475+0000", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0840000000000003e-06 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534924+0000", 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044986499999999998 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540008+0000", 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050840019999999998 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=63, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65316706800000002, 2024-09-15T01:44:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058134440000000001, 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.13e-07 2024-09-15T01:44:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534542+0000", 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3890000000000002e-06 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534557+0000", 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4919000000000001e-05 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534562+0000", 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7189999999999998e-06 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540154+0000", 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055920869999999999 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540348+0000", 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019381699999999999 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=64, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65313378200000005, 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059416290000000004, 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534573+0000", 2024-09-15T01:44:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534579+0000", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6479999999999997e-06 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534584+0000", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6079999999999998e-06 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540397+0000", 2024-09-15T01:44:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058133880000000001 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540510+0000", 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001125 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.8s1 33/32 ECSubReadReply(tid=26, attrs_read=0))", 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534714+0000", 2024-09-15T01:44:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65298803699999997, 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030491089999999999, 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534714+0000", 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534714+0000", 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534714+0000", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7500000000000001e-07 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534787+0000", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2682999999999995e-05 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534794+0000", 2024-09-15T01:44:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5780000000000001e-06 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534799+0000", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8040000000000001e-06 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535859+0000", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001059824 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537763+0000", 2024-09-15T01:44:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001903945 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65225475899999996, 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-09-15T01:44:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.65098987900000005, 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041052659999999998, 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536713+0000", 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8999999999999996e-07 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536737+0000", 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3980000000000001e-05 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536757+0000", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0792e-05 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536767+0000", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4409999999999999e-06 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536814+0000", 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7348000000000001e-05 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536825+0000", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0723999999999999e-05 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540785+0000", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039597180000000001 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540817+0000", 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2573000000000001e-05 2024-09-15T01:44:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s7 33/32 ECSubWrite(tid=30, reqid=client.4632.0:211, at_version=33'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547748+0000", 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.63995353899999996, 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030008859999999999, 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547748+0000", 2024-09-15T01:44:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547748+0000", 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547749+0000", 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7800000000000001e-07 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547812+0000", 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2568000000000001e-05 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547837+0000", 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5758e-05 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547845+0000", 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.554e-06 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547933+0000", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7984000000000003e-05 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547950+0000", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6619000000000001e-05 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550720+0000", 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027706689999999999 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550749+0000", 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9056000000000001e-05 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.295 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:13.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:13.297 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:13.292+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:13.397 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:13.392+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:13.403 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:13.403 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:13.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:13.499 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:13.492+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:73: done (11 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:74: done (10 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:75: done (9 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:77: done (8 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:78: done (7 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:79: done (6 left) 2024-09-15T01:44:13.520 INFO:tasks.rados.rados.0.smithi033.stdout:81: done (5 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:82: done (4 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:83: done (3 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:84: done (2 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:85: done (1 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:86: done (0 left) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:87: read oid 39 snap 0 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:87: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:88: rollback oid 47 current snap is 3 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 47 to 0 2024-09-15T01:44:13.521 INFO:tasks.rados.rados.0.smithi033.stdout:89: snap_create 2024-09-15T01:44:13.526 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:44:13.528 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:13.529 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:13.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:13.600 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:13.596+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:13.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:13.634 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4317.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.232253541999999, 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087692900000000003, 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2399999999999999e-07 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397131+0000", 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9860000000000002e-06 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397145+0000", 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4652e-05 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397160+0000", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4124000000000001e-05 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397218+0000", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7942999999999997e-05 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397265+0000", 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6953999999999998e-05 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397961+0000", 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00069677499999999996 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.398002+0000", 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1171000000000003e-05 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.8s2 33/32 ECSubRead(tid=24, to_read={3:1768942e:::smithi03327189-19:head=0,77824,0}, subchunks={3:1768942e:::smithi03327189-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.524805+0000", 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.104573952, 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077387799999999998, 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524805+0000", 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524805+0000", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524806+0000", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.009e-06 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524815+0000", 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8119999999999997e-06 2024-09-15T01:44:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524831+0000", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6453999999999999e-05 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524839+0000", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0989999999999999e-06 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.524912+0000", 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2088000000000005e-05 2024-09-15T01:44:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525579+0000", 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066741599999999999 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.103357672, 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003107659, 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526022+0000", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526051+0000", 2024-09-15T01:44:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8187e-05 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526061+0000", 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0343000000000001e-05 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526068+0000", 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7469999999999999e-06 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526304+0000", 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000236197 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526315+0000", 2024-09-15T01:44:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0946e-05 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529101+0000", 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027858359999999999 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529129+0000", 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8313000000000001e-05 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.8s2 33/32 ECSubRead(tid=25, to_read={3:1768942e:::smithi03327189-19:head=0,77824,0}, subchunks={3:1768942e:::smithi03327189-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.529395+0000", 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0999840679999999, 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046363099999999999, 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529395+0000", 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529395+0000", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529396+0000", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5600000000000004e-07 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529407+0000", 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1385e-05 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529451+0000", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3645e-05 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529460+0000", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.799e-06 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529735+0000", 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027554899999999999 2024-09-15T01:44:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529859+0000", 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000123297 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.8s2 33/32 ECSubRead(tid=26, to_read={3:1768942e:::smithi03327189-19:head=0,77824,0}, subchunks={3:1768942e:::smithi03327189-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.529479+0000", 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0999000459999999, 2024-09-15T01:44:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00070429599999999998, 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529479+0000", 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529479+0000", 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529479+0000", 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.39e-07 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529486+0000", 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2219999999999999e-06 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529494+0000", 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6840000000000002e-06 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529500+0000", 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9689999999999999e-06 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529952+0000", 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045117200000000001 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530183+0000", 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023180999999999999 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0938869229999999, 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.es0 33/32 [PushOp(3:720ccbad:::smithi03327189-29:head, version: 30'2, data_included: [0~118784], data_size: 118784, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:720ccbad:::smithi03327189-29:head@30'2, size: 475136, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:475136, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0886221620000001, 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00112323, 2024-09-15T01:44:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541186+0000", 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042845600000000002 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541204+0000", 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7867999999999999e-05 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541215+0000", 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1609e-05 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541347+0000", 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000132008 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541880+0000", 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000532962 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.7s1 33/32 [PushOp(3:e5da12c9:::smithi03327189-25:head, version: 30'5, data_included: [0~102400], data_size: 102400, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:e5da12c9:::smithi03327189-25:head@30'5, size: 409600, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:409600, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544193+0000", 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0851862400000001, 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00081221400000000001, 2024-09-15T01:44:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544193+0000", 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544193+0000", 2024-09-15T01:44:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544193+0000", 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7399999999999999e-07 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544369+0000", 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017561000000000001 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544390+0000", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1132e-05 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544400+0000", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0125e-05 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544502+0000", 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010194100000000001 2024-09-15T01:44:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545005+0000", 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050313199999999999 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/32 ECSubWrite(tid=66, reqid=client.4632.0:197, at_version=33'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.080620962, 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026146149999999998, 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548760+0000", 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.663e-06 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549278+0000", 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051834099999999996 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549296+0000", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7742000000000001e-05 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549304+0000", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3620000000000006e-06 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549384+0000", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0055999999999994e-05 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549396+0000", 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1759999999999999e-05 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551342+0000", 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001946138 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551373+0000", 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0552999999999997e-05 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.es0 33/32 [PushOp(3:7de15fb4:::smithi03327189-49:head, version: 30'5, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:7de15fb4:::smithi03327189-49:head@30'5, size: 819200, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:819200, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.079890604, 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00108639, 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549489+0000", 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.23e-07 2024-09-15T01:44:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549872+0000", 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038235 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549900+0000", 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8373000000000001e-05 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549913+0000", 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3258000000000001e-05 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550006+0000", 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2798000000000002e-05 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550575+0000", 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056878799999999998 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.3s4 33/32 [PushOp(3:cad9bd25:::smithi03327189-12:head, version: 30'2, data_included: [0~49152], data_size: 49152, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:cad9bd25:::smithi03327189-12:head@30'2, size: 196608, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:196608, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.552667+0000", 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0767116350000001, 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066175800000000003, 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552667+0000", 2024-09-15T01:44:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552667+0000", 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552668+0000", 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7300000000000002e-07 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552901+0000", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023314599999999999 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552919+0000", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8425999999999999e-05 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552926+0000", 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6760000000000001e-06 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552998+0000", 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1971999999999995e-05 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.553329+0000", 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00033116500000000002 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.7s1 33/32 [PushOp(3:e02ccbe4:::smithi03327189-36:head, version: 30'8, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e02ccbe4:::smithi03327189-36:head@30'8, size: 1490944, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0737079549999999, 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010742620000000001, 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555672+0000", 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.540000000000001e-07 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555831+0000", 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000158868 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555856+0000", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4610999999999999e-05 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555868+0000", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2492e-05 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555951+0000", 2024-09-15T01:44:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2587000000000001e-05 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556745+0000", 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079474999999999999 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.3s4 33/32 [PushOp(3:c5b7678e:::smithi03327189-28:head, version: 30'4, data_included: [0~114688], data_size: 114688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c5b7678e:::smithi03327189-28:head@30'4, size: 458752, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:458752, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0711906369999999, 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001032888, 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558189+0000", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3399999999999998e-07 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558666+0000", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00047639900000000002 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558683+0000", 2024-09-15T01:44:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7101000000000001e-05 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558692+0000", 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5559999999999996e-06 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558781+0000", 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9085999999999994e-05 2024-09-15T01:44:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559221+0000", 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044001199999999999 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/32 ECSubWrite(tid=69, reqid=client.4632.0:197, at_version=0'0, trim_to=0'0, roll_forward_to=33'12))", 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.070871184, 2024-09-15T01:44:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00098901999999999996, 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558509+0000", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558529+0000", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0579000000000001e-05 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558557+0000", 2024-09-15T01:44:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7719999999999999e-05 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558572+0000", 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5607000000000002e-05 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558627+0000", 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5025000000000003e-05 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558639+0000", 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1578e-05 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559471+0000", 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00083245300000000003 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559497+0000", 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5437999999999999e-05 2024-09-15T01:44:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.7s1 33/32 [PushOp(3:e02ccbe4:::smithi03327189-36:head, version: 30'8, data_included: [262144~110592], data_size: 110592, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:e02ccbe4:::smithi03327189-36:head@30'8, size: 1490944, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1490944, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.561461+0000", 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.06791787, 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063459200000000001, 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561461+0000", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561461+0000", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561462+0000", 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0330000000000001e-06 2024-09-15T01:44:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561509+0000", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7216000000000003e-05 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561531+0000", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1719999999999999e-05 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561542+0000", 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1229e-05 2024-09-15T01:44:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561679+0000", 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000136119 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562096+0000", 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041727499999999999 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.cs5 33/32 ECSubRead(tid=72, to_read={3:3cf1f308:::smithi03327189-35:2=0,262144,0}, subchunks={3:3cf1f308:::smithi03327189-35:2=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.568818+0000", 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0605607479999999, 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00071021599999999995, 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568818+0000", 2024-09-15T01:44:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568818+0000", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568819+0000", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1349999999999999e-06 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568834+0000", 2024-09-15T01:44:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4924e-05 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568859+0000", 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4413000000000002e-05 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568873+0000", 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4355999999999999e-05 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.568948+0000", 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4618999999999998e-05 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.569529+0000", 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058076899999999997 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.cs5 33/32 ECSubRead(tid=74, to_read={3:3cf1f308:::smithi03327189-35:head=0,262144,0}, subchunks={3:3cf1f308:::smithi03327189-35:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.055385432, 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010707749999999999, 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573995+0000", 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2810000000000001e-06 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574023+0000", 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7721000000000001e-05 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574065+0000", 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1881000000000001e-05 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574074+0000", 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7620000000000007e-06 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574152+0000", 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7248000000000003e-05 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.575064+0000", 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091288199999999995 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=81, reqid=client.4632.0:200, at_version=33'23, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.021101606, 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001931767, 2024-09-15T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608279+0000", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5969999999999999e-06 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608325+0000", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5448000000000003e-05 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608364+0000", 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9894000000000001e-05 2024-09-15T01:44:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608379+0000", 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4249e-05 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608512+0000", 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013341699999999999 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608534+0000", 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2113e-05 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610148+0000", 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016133300000000001 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610209+0000", 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1718999999999996e-05 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=82, reqid=client.4632.0:201, at_version=33'24, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0185664329999999, 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001354173, 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610814+0000", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3200000000000001e-06 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610857+0000", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3056000000000002e-05 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610886+0000", 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8645000000000001e-05 2024-09-15T01:44:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610899+0000", 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3173000000000001e-05 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610982+0000", 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3129999999999999e-05 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.611003+0000", 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.126e-05 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.612129+0000", 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011261750000000001 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.612167+0000", 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7413999999999998e-05 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=83, reqid=client.4632.0:201, at_version=0'0, trim_to=0'0, roll_forward_to=33'24))", 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.014378421, 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001080248, 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0200000000000001e-07 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615035+0000", 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3966e-05 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615052+0000", 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615060+0000", 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2740000000000002e-06 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615145+0000", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4999000000000004e-05 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615168+0000", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3722000000000001e-05 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.616041+0000", 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087272599999999999 2024-09-15T01:44:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.616081+0000", 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9888999999999997e-05 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:13.719 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:13.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:13.721 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:13.716+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:13.760 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:13.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:13.801 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:13.821 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:13.821 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:13.821 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:13.821 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:13.816+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:13.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:13.844 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:13.871 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:13.907 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:13.921 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:13.915+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:13.936 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:13.952 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:13.952 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:13.957 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:13.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:13.987 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:14.007 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:14.022 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:14.016+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:14.029 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:14.032 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:14.086 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.086 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:14.123 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:14.116+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:14.206 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.207 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:14.224 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:14.219+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:14.320 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.320 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:14.320+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:14.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:14.426 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:14.420+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:14.429 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.429 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:14.526 INFO:tasks.rados.rados.0.smithi033.stdout:89: done (2 left) 2024-09-15T01:44:14.526 INFO:tasks.rados.rados.0.smithi033.stdout:90: setattr oid 21 current snap is 4 2024-09-15T01:44:14.526 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 7 current snap is 4 2024-09-15T01:44:14.526 INFO:tasks.rados.rados.0.smithi033.stdout:91: seq_num 61 ranges {114688=655360,770048=376832} 2024-09-15T01:44:14.527 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:14.520+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:14.532 INFO:tasks.rados.rados.0.smithi033.stdout:91: writing smithi03327189-7 from 114688 to 770048 tid 1 2024-09-15T01:44:14.536 INFO:tasks.rados.rados.0.smithi033.stdout:91: writing smithi03327189-7 from 770048 to 1146880 tid 2 2024-09-15T01:44:14.536 INFO:tasks.rados.rados.0.smithi033.stdout:92: snap_create 2024-09-15T01:44:14.544 INFO:tasks.rados.rados.0.smithi033.stdout:87: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-15T01:44:14.563 INFO:tasks.rados.rados.0.smithi033.stdout:88: finishing rollback tid 0 to smithi03327189-47 2024-09-15T01:44:14.563 INFO:tasks.rados.rados.0.smithi033.stdout:88: finishing rollback tid 1 to smithi03327189-47 2024-09-15T01:44:14.563 INFO:tasks.rados.rados.0.smithi033.stdout:88: finishing rollback tid 2 to smithi03327189-47 2024-09-15T01:44:14.563 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 28 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:91: finishing write tid 1 to smithi03327189-7 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:91: finishing write tid 2 to smithi03327189-7 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:91: finishing write tid 3 to smithi03327189-7 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:91: finishing write tid 4 to smithi03327189-7 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 10 (ObjNum 61 snap 4 seq_num 61) dirty exists 2024-09-15T01:44:14.564 INFO:tasks.rados.rados.0.smithi033.stdout:91: left oid 7 (ObjNum 61 snap 4 seq_num 61) 2024-09-15T01:44:14.599 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.599 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:14.628 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:14.620+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:14.765 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:14.760+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.822 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.829 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:14.823+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:14.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:14.930 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:14.924+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:14.933 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:14.933 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:14.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:15.031 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:15.024+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:15.041 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.041 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:15.133 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:15.128+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:15.153 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.153 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:15.233 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:15.228+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:15.268 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.268 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:15.334 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:15.328+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:15.378 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.378 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:15.434 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:15.428+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:15.493 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.493 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:15.512 INFO:tasks.rados.rados.0.smithi033.stdout:87: done (4 left) 2024-09-15T01:44:15.512 INFO:tasks.rados.rados.0.smithi033.stdout:88: done (3 left) 2024-09-15T01:44:15.513 INFO:tasks.rados.rados.0.smithi033.stdout:90: done (2 left) 2024-09-15T01:44:15.513 INFO:tasks.rados.rados.0.smithi033.stdout:91: done (1 left) 2024-09-15T01:44:15.513 INFO:tasks.rados.rados.0.smithi033.stdout:92: done (0 left) 2024-09-15T01:44:15.513 INFO:tasks.rados.rados.0.smithi033.stdout:93: delete oid 17 current snap is 5 2024-09-15T01:44:15.521 INFO:tasks.rados.rados.0.smithi033.stdout:93: done (0 left) 2024-09-15T01:44:15.521 INFO:tasks.rados.rados.0.smithi033.stdout:94: snap_remove snap 2 2024-09-15T01:44:15.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:44:15.545 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:15.540+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:15.607 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] out_osds: [11] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:15.607 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:15.607 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T01:44:15.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T01:44:15.608 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.608 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.616 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:15.636 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:15.632+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:15.717 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.717 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.729 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:15.738 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:15.732+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:15.825 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:15.825 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:15.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:15.838 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:15.836+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:15.906 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 36, 2024-09-15T01:44:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "d386169c-ec95-4076-a6ad-eaab3ccf750b", 2024-09-15T01:44:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-09-15T01:43:41.737992+0000", 2024-09-15T01:44:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-09-15T01:44:15.502418+0000", 2024-09-15T01:44:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "last_up_change": "2024-09-15T01:43:52.350040+0000", 2024-09-15T01:44:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "last_in_change": "2024-09-15T01:44:10.556061+0000", 2024-09-15T01:44:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T01:44:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_num": 5799936, 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_set": [ 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "pglog_hardlimit", 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "purged_snapdirs", 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_deletes", 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "sortbitwise" 2024-09-15T01:44:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_version": 6, 2024-09-15T01:44:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T01:44:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T01:44:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T01:44:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_snapshot": "", 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_max": 3, 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd": 12, 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "require_min_compat_client": "luminous", 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min_compat_client": "jewel", 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "require_osd_release": "quincy", 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "pools": [ 2024-09-15T01:44:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 1, 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": ".mgr", 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:52.349089+0000", 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool", 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:44:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 1, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 1, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 1, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 1, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 1, 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "20", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 0, 2024-09-15T01:44:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 0, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:44:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:44:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "options": { 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_max": 32, 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_min": 1 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": {} 2024-09-15T01:44:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 2, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "rbd", 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:54.322526+0000", 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:44:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 8, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 8, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 8, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 8, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 8, 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "23", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 3, 2024-09-15T01:44:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 23, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd": {} 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "unique_pool_0", 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:44:05.562069+0000", 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 3, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 8, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 5, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 1, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 16, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 16, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 16, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 16, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 16, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "36", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 7, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 36, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "lrcprofile", 2024-09-15T01:44:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 16384, 2024-09-15T01:44:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "rados": {} 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "5bbf5b49-bace-424f-b96c-91c3a25e48e0", 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6802", 2024-09-15T01:44:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6803", 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6806", 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6807", 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6804", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6805", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6801/22087", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6803/22087", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6807/22087", 2024-09-15T01:44:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6805/22087", 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464", 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6808", 2024-09-15T01:44:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6809", 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6810", 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6811", 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6814", 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6815", 2024-09-15T01:44:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6812", 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6813", 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6809/22086", 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6811/22086", 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6815/22086", 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6813/22086", 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "fcc939b7-c3fb-4117-af41-9eec226f3020", 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6816", 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6817", 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6818", 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6819", 2024-09-15T01:44:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6822", 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6823", 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6820", 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6821", 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6817/22088", 2024-09-15T01:44:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6819/22088", 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6823/22088", 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6821/22088", 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "57f1d668-f85b-4b0c-9093-fb5fef4b91ef", 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6824", 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6825", 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6826", 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6827", 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6830", 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6831", 2024-09-15T01:44:15.959 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6828", 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6829", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6825/22089", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6827/22089", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6831/22089", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6829/22089", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.961 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "a274d12c-8683-491d-8656-5af5d12164ee", 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.962 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6800", 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6801", 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6802", 2024-09-15T01:44:15.964 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6803", 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6806", 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6807", 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6804", 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6805", 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.967 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6801/22096", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6803/22096", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6807/22096", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6805/22096", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "f0a90a47-9b16-46f6-a2a7-afa4a6f2a298", 2024-09-15T01:44:15.968 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6816", 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6817", 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.970 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6818", 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6819", 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6822", 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6823", 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6820", 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6821", 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6817/22097", 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6819/22097", 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6823/22097", 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6821/22097", 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.975 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "194e23d1-7f52-43ee-8557-52d75a3c22dc", 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 27, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6808", 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6809", 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6810", 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6811", 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6814", 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6815", 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6812", 2024-09-15T01:44:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6813", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6809/22108", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6811/22108", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6815/22108", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6813/22108", 2024-09-15T01:44:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1d1c61e7-be03-4c72-bc95-e187eb95a1b8", 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 18, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6824", 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6825", 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6826", 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6827", 2024-09-15T01:44:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6830", 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6831", 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6828", 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6829", 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6825/22109", 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6827/22109", 2024-09-15T01:44:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6831/22109", 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6829/22109", 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1f23463d-2214-416b-b3ba-23132f2d84ee", 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 0, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 33, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6801", 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6803", 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6805", 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6807", 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6813", 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6815", 2024-09-15T01:44:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6809", 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6811", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6803/22357", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6807/22357", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6815/22357", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6811/22357", 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "39226fc7-a654-41c6-9d70-5ea2f69c236b", 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 33, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6800", 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6802", 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6804", 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6806", 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6812", 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:15.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6814", 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6808", 2024-09-15T01:44:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6810", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6802/22359", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6806/22359", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6814/22359", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6810/22359", 2024-09-15T01:44:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "eaafc17e-407d-43aa-8083-2c56660d7db4", 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 0, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6816", 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6817", 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6818", 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6819", 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6822", 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6823", 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6820", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6821", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6817/22358", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6819/22358", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6823/22358", 2024-09-15T01:44:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6821/22358", 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "05fdd01d-8c78-4a61-81e7-1d9816e951a3", 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 0, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 27, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6824", 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6825", 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6826", 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6827", 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6830", 2024-09-15T01:44:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6831", 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6828", 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6829", 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6825/22360", 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6827/22360", 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6831/22360", 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6829/22360", 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_xinfo": [ 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.101167+0000", 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.166075+0000", 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.012320+0000", 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.227835+0000", 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.010828+0000", 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.170791+0000", 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:44:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.037098+0000", 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.281668+0000", 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:49.661197+0000", 2024-09-15T01:44:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.009199+0000", 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:44:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.143021+0000", 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 65536, 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.187329+0000", 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap": [], 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap_items": [], 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_temp": [], 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_temp": [], 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklist": {}, 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "range_blocklist": {}, 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profiles": { 2024-09-15T01:44:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default": { 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "2", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "1", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "jerasure", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "technique": "reed_sol_van" 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "lrcprofile": { 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-device-class": "", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-root": "default", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "4", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "l": "3", 2024-09-15T01:44:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "2", 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "lrcprofile", 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "lrc" 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps_queue": [ 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "snaps": [ 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 3, 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 1 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "new_removed_snaps": [], 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "new_purged_snaps": [], 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_node_flags": {}, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "device_class_flags": {}, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": { 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_enabled": false, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_bucket_count": 0, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "degraded_stretch_mode": 0, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "recovering_stretch_mode": 0, 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_bucket": 0 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:16.032 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:16.034 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:16.034 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:16.035 INFO:tasks.thrashosds.thrasher:j is {'epoch': 36, 'fsid': 'd386169c-ec95-4076-a6ad-eaab3ccf750b', 'created': '2024-09-15T01:43:41.737992+0000', 'modified': '2024-09-15T01:44:15.502418+0000', 'last_up_change': '2024-09-15T01:43:52.350040+0000', 'last_in_change': '2024-09-15T01:44:10.556061+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:43:52.349089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:43:54.322526+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T01:44:05.562069+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 8, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'lrcprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '5bbf5b49-bace-424f-b96c-91c3a25e48e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 22087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 22087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 22087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6805', 'nonce': 22087}]}, 'public_addr': '172.21.15.33:6801/22087', 'cluster_addr': '172.21.15.33:6803/22087', 'heartbeat_back_addr': '172.21.15.33:6807/22087', 'heartbeat_front_addr': '172.21.15.33:6805/22087', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6808', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6809', 'nonce': 22086}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6810', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 22086}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6814', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 22086}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6813', 'nonce': 22086}]}, 'public_addr': '172.21.15.33:6809/22086', 'cluster_addr': '172.21.15.33:6811/22086', 'heartbeat_back_addr': '172.21.15.33:6815/22086', 'heartbeat_front_addr': '172.21.15.33:6813/22086', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fcc939b7-c3fb-4117-af41-9eec226f3020', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6817', 'nonce': 22088}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6818', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 22088}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 22088}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 22088}]}, 'public_addr': '172.21.15.33:6817/22088', 'cluster_addr': '172.21.15.33:6819/22088', 'heartbeat_back_addr': '172.21.15.33:6823/22088', 'heartbeat_front_addr': '172.21.15.33:6821/22088', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57f1d668-f85b-4b0c-9093-fb5fef4b91ef', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6824', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6825', 'nonce': 22089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6826', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6827', 'nonce': 22089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6830', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6831', 'nonce': 22089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6828', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6829', 'nonce': 22089}]}, 'public_addr': '172.21.15.33:6825/22089', 'cluster_addr': '172.21.15.33:6827/22089', 'heartbeat_back_addr': '172.21.15.33:6831/22089', 'heartbeat_front_addr': '172.21.15.33:6829/22089', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a274d12c-8683-491d-8656-5af5d12164ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6800', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6801', 'nonce': 22096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6802', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6803', 'nonce': 22096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6806', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6807', 'nonce': 22096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6804', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6805', 'nonce': 22096}]}, 'public_addr': '172.21.15.47:6801/22096', 'cluster_addr': '172.21.15.47:6803/22096', 'heartbeat_back_addr': '172.21.15.47:6807/22096', 'heartbeat_front_addr': '172.21.15.47:6805/22096', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f0a90a47-9b16-46f6-a2a7-afa4a6f2a298', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6816', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6817', 'nonce': 22097}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6818', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6819', 'nonce': 22097}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6822', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6823', 'nonce': 22097}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6820', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6821', 'nonce': 22097}]}, 'public_addr': '172.21.15.47:6817/22097', 'cluster_addr': '172.21.15.47:6819/22097', 'heartbeat_back_addr': '172.21.15.47:6823/22097', 'heartbeat_front_addr': '172.21.15.47:6821/22097', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '194e23d1-7f52-43ee-8557-52d75a3c22dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6808', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6809', 'nonce': 22108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6810', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6811', 'nonce': 22108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6814', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6815', 'nonce': 22108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6812', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6813', 'nonce': 22108}]}, 'public_addr': '172.21.15.47:6809/22108', 'cluster_addr': '172.21.15.47:6811/22108', 'heartbeat_back_addr': '172.21.15.47:6815/22108', 'heartbeat_front_addr': '172.21.15.47:6813/22108', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '1d1c61e7-be03-4c72-bc95-e187eb95a1b8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6824', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6825', 'nonce': 22109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6826', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6827', 'nonce': 22109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6830', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6831', 'nonce': 22109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6828', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6829', 'nonce': 22109}]}, 'public_addr': '172.21.15.47:6825/22109', 'cluster_addr': '172.21.15.47:6827/22109', 'heartbeat_back_addr': '172.21.15.47:6831/22109', 'heartbeat_front_addr': '172.21.15.47:6829/22109', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '1f23463d-2214-416b-b3ba-23132f2d84ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6801', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6803', 'nonce': 22357}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6805', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6807', 'nonce': 22357}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6813', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6815', 'nonce': 22357}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6809', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6811', 'nonce': 22357}]}, 'public_addr': '172.21.15.155:6803/22357', 'cluster_addr': '172.21.15.155:6807/22357', 'heartbeat_back_addr': '172.21.15.155:6815/22357', 'heartbeat_front_addr': '172.21.15.155:6811/22357', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '39226fc7-a654-41c6-9d70-5ea2f69c236b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6800', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6802', 'nonce': 22359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6804', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6806', 'nonce': 22359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6812', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6814', 'nonce': 22359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6808', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6810', 'nonce': 22359}]}, 'public_addr': '172.21.15.155:6802/22359', 'cluster_addr': '172.21.15.155:6806/22359', 'heartbeat_back_addr': '172.21.15.155:6814/22359', 'heartbeat_front_addr': '172.21.15.155:6810/22359', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'eaafc17e-407d-43aa-8083-2c56660d7db4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6816', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6817', 'nonce': 22358}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6818', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6819', 'nonce': 22358}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6822', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6823', 'nonce': 22358}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6820', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6821', 'nonce': 22358}]}, 'public_addr': '172.21.15.155:6817/22358', 'cluster_addr': '172.21.15.155:6819/22358', 'heartbeat_back_addr': '172.21.15.155:6823/22358', 'heartbeat_front_addr': '172.21.15.155:6821/22358', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '05fdd01d-8c78-4a61-81e7-1d9816e951a3', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6824', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6825', 'nonce': 22360}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6826', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6827', 'nonce': 22360}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6830', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6831', 'nonce': 22360}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6828', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6829', 'nonce': 22360}]}, 'public_addr': '172.21.15.155:6825/22360', 'cluster_addr': '172.21.15.155:6827/22360', 'heartbeat_back_addr': '172.21.15.155:6831/22360', 'heartbeat_front_addr': '172.21.15.155:6829/22360', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.101167+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.166075+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.012320+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.227835+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.010828+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.170791+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.037098+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.281668+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:49.661197+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.009199+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.143021+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.187329+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'lrcprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'l': '3', 'm': '2', 'name': 'lrcprofile', 'plugin': 'lrc'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 3, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T01:44:16.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:16.036 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:16.037 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:16.028+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:16.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:16.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:16.128+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:16.143 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:16.143 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:16.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:16.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:16.235 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:16.228+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:16.240 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:16.254 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:16.254 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:16.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:16.282 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:16.302 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:16.325 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:16.336 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:16.327+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:16.344 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:16.347 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:16.358 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-15T01:44:16.288514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19579443,"num_objects":42,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":166,"num_read_kb":2264,"num_write":265,"num_write_kb":22744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2888224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":146,"acting":145,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":11,"num_per_pool_osds":11,"num_per_pool_omap_osds":11,"kb":1038090240,"kb_used":3315036,"kb_used_data":44112,"kb_used_omap":18,"kb_used_meta":3270765,"kb_avail":1034775204,"statfs":{"total":1063004405760,"available":1059609808896,"internally_reserved":0,"allocated":45170688,"data_stored":43332267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18744,"internal_metadata":3349264072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2359296,"num_objects":6,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":2167,"num_write":13,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2888224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001310"},"pg_stats":[{"pgid":"3.f","version":"30'7","reported_seq":59,"reported_epoch":33,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:44:12.562352+0000","last_change":"2024-09-15T01:44:12.562963+0000","last_active":"2024-09-15T01:44:12.562352+0000","last_peered":"2024-09-15T01:44:12.562352+0000","last_clean":"2024-09-15T01:44:09.798622+0000","last_became_active":"2024-09-15T01:44:12.526254+0000","last_became_peered":"2024-09-15T01:44:12.526254+0000","last_unstale":"2024-09-15T01:44:12.562352+0000","last_undegraded":"2024-09-15T01:44:12.562352+0000","last_fullsized":"2024-09-15T01:44:12.509707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.046152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2165,"num_write":7,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,2147483647,8,2,9,10,0,4],"avail_no_missing":["7(0)","0(6)","1(1)","2(3)","4(7)","8(2)","9(4)","10(5)","11(1)"],"object_location_counts":[{"shards":"0(6),2(3),4(7),7(0),8(2),9(4),10(5)","objects":3}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"peering","last_fresh":"0.000000","last_change":"2024-09-15T01:44:10.583672+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"0.000000","last_undegraded":"0.000000","last_fullsized":"0.000000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:10.447544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,4,8,9,10,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'6","reported_seq":52,"reported_epoch":33,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:44:12.574142+0000","last_change":"2024-09-15T01:44:12.574722+0000","last_active":"2024-09-15T01:44:12.574142+0000","last_peered":"2024-09-15T01:44:12.574142+0000","last_clean":"2024-09-15T01:44:09.700560+0000","last_became_active":"2024-09-15T01:44:12.536252+0000","last_became_peered":"2024-09-15T01:44:12.536252+0000","last_unstale":"2024-09-15T01:44:12.574142+0000","last_undegraded":"2024-09-15T01:44:12.574142+0000","last_fullsized":"2024-09-15T01:44:12.509683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.174235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1540096,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,1,3,4,0],"acting":[7,5,9,6,2147483647,3,4,0],"avail_no_missing":["7(0)","0(7)","1(4)","3(5)","4(6)","5(1)","6(3)","9(2)","11(4)"],"object_location_counts":[{"shards":"0(7),3(5),4(6),5(1),6(3),7(0),9(2)","objects":3}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":34,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.575273+0000","last_change":"2024-09-15T01:44:10.575738+0000","last_active":"2024-09-15T01:44:10.575214+0000","last_peered":"2024-09-15T01:44:09.839928+0000","last_clean":"2024-09-15T01:44:09.839928+0000","last_became_active":"2024-09-15T01:44:07.709221+0000","last_became_peered":"2024-09-15T01:44:07.709221+0000","last_unstale":"2024-09-15T01:44:10.575273+0000","last_undegraded":"2024-09-15T01:44:10.575273+0000","last_fullsized":"2024-09-15T01:44:10.575273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:30.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":10,"num_write_kb":2192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,7],"acting":[5,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,3,6,7,9,10,11],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"30'4","reported_seq":28,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.584549+0000","last_change":"2024-09-15T01:44:10.585044+0000","last_active":"2024-09-15T01:44:10.584513+0000","last_peered":"2024-09-15T01:44:09.792997+0000","last_clean":"2024-09-15T01:44:09.792997+0000","last_became_active":"2024-09-15T01:44:07.563862+0000","last_became_peered":"2024-09-15T01:44:07.563862+0000","last_unstale":"2024-09-15T01:44:10.584549+0000","last_undegraded":"2024-09-15T01:44:10.584549+0000","last_fullsized":"2024-09-15T01:44:10.584549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.054874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,4,6,7,8,9,11],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"30'6","reported_seq":26,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.719037+0000","last_change":"2024-09-15T01:44:07.580942+0000","last_active":"2024-09-15T01:44:09.719037+0000","last_peered":"2024-09-15T01:44:09.719037+0000","last_clean":"2024-09-15T01:44:09.719037+0000","last_became_active":"2024-09-15T01:44:07.579889+0000","last_became_peered":"2024-09-15T01:44:07.579889+0000","last_unstale":"2024-09-15T01:44:09.719037+0000","last_undegraded":"2024-09-15T01:44:09.719037+0000","last_fullsized":"2024-09-15T01:44:09.719037+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.248888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"peering","last_fresh":"0.000000","last_change":"2024-09-15T01:44:10.584137+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"0.000000","last_undegraded":"0.000000","last_fullsized":"0.000000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:22.587024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,5,1,4,10],"acting":[8,3,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,5,7,8,9,10,11],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"30'4","reported_seq":23,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.574676+0000","last_change":"2024-09-15T01:44:10.575675+0000","last_active":"2024-09-15T01:44:10.574582+0000","last_peered":"2024-09-15T01:44:09.713390+0000","last_clean":"2024-09-15T01:44:09.713390+0000","last_became_active":"2024-09-15T01:44:07.705410+0000","last_became_peered":"2024-09-15T01:44:07.705410+0000","last_unstale":"2024-09-15T01:44:10.574676+0000","last_undegraded":"2024-09-15T01:44:10.574676+0000","last_fullsized":"2024-09-15T01:44:10.574676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:13.861681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1032192,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,5,10,9],"acting":[8,1,2,4,3,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,4,5,8,9,10,11],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":149,"reported_epoch":33,"state":"active+undersized+remapped","last_fresh":"2024-09-15T01:44:12.529480+0000","last_change":"2024-09-15T01:44:12.529976+0000","last_active":"2024-09-15T01:44:12.529480+0000","last_peered":"2024-09-15T01:44:12.529480+0000","last_clean":"2024-09-15T01:44:09.566043+0000","last_became_active":"2024-09-15T01:44:12.519264+0000","last_became_peered":"2024-09-15T01:44:12.519264+0000","last_unstale":"2024-09-15T01:44:12.529480+0000","last_undegraded":"2024-09-15T01:44:12.529480+0000","last_fullsized":"2024-09-15T01:44:12.513354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:57.506567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":["7","6","11"],"object_location_counts":[{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.560275+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:44:09.560275+0000","last_peered":"2024-09-15T01:44:09.560275+0000","last_clean":"2024-09-15T01:44:09.560275+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:44:09.560275+0000","last_undegraded":"2024-09-15T01:44:09.560275+0000","last_fullsized":"2024-09-15T01:44:09.560275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"30'5","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.844218+0000","last_change":"2024-09-15T01:44:07.580316+0000","last_active":"2024-09-15T01:44:09.844218+0000","last_peered":"2024-09-15T01:44:09.844218+0000","last_clean":"2024-09-15T01:44:09.844218+0000","last_became_active":"2024-09-15T01:44:07.579340+0000","last_became_peered":"2024-09-15T01:44:07.579340+0000","last_unstale":"2024-09-15T01:44:09.844218+0000","last_undegraded":"2024-09-15T01:44:09.844218+0000","last_fullsized":"2024-09-15T01:44:09.844218+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.070058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540672,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-15T01:44:11.507701+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:44:11.507701+0000","last_peered":"2024-09-15T01:44:11.507701+0000","last_clean":"2024-09-15T01:44:11.507701+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:44:11.507701+0000","last_undegraded":"2024-09-15T01:44:11.507701+0000","last_fullsized":"2024-09-15T01:44:11.507701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"30'6","reported_seq":28,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.578527+0000","last_change":"2024-09-15T01:44:10.580265+0000","last_active":"2024-09-15T01:44:10.578484+0000","last_peered":"2024-09-15T01:44:09.604605+0000","last_clean":"2024-09-15T01:44:09.604605+0000","last_became_active":"2024-09-15T01:44:07.579175+0000","last_became_peered":"2024-09-15T01:44:07.579175+0000","last_unstale":"2024-09-15T01:44:10.578527+0000","last_undegraded":"2024-09-15T01:44:10.578527+0000","last_fullsized":"2024-09-15T01:44:10.578527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.287774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442368,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,5,9,6,1],"acting":[0,4,10,3,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,4,5,6,9,10,11],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.571570+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:44:09.571570+0000","last_peered":"2024-09-15T01:44:09.571570+0000","last_clean":"2024-09-15T01:44:09.571570+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:44:09.571570+0000","last_undegraded":"2024-09-15T01:44:09.571570+0000","last_fullsized":"2024-09-15T01:44:09.571570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"peering","last_fresh":"0.000000","last_change":"2024-09-15T01:44:10.577700+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"0.000000","last_undegraded":"0.000000","last_fullsized":"0.000000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.851239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,1,7,8,0,3],"acting":[5,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,5,6,7,8,9,11],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":25,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.560238+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:44:09.560238+0000","last_peered":"2024-09-15T01:44:09.560238+0000","last_clean":"2024-09-15T01:44:09.560238+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:44:09.560238+0000","last_undegraded":"2024-09-15T01:44:09.560238+0000","last_fullsized":"2024-09-15T01:44:09.560238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'4","reported_seq":24,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.585913+0000","last_change":"2024-09-15T01:44:10.586301+0000","last_active":"2024-09-15T01:44:10.585877+0000","last_peered":"2024-09-15T01:44:09.649499+0000","last_clean":"2024-09-15T01:44:09.649499+0000","last_became_active":"2024-09-15T01:44:07.577035+0000","last_became_peered":"2024-09-15T01:44:07.577035+0000","last_unstale":"2024-09-15T01:44:10.585913+0000","last_undegraded":"2024-09-15T01:44:10.585913+0000","last_fullsized":"2024-09-15T01:44:10.585913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:28.699712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,5,9,10],"acting":[4,3,0,6,2,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,3,5,6,9,10,11],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"30'20","reported_seq":49,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.574016+0000","last_change":"2024-09-15T01:44:10.574468+0000","last_active":"2024-09-15T01:44:10.573979+0000","last_peered":"2024-09-15T01:44:09.795369+0000","last_clean":"2024-09-15T01:44:09.795369+0000","last_became_active":"2024-09-15T01:44:07.707040+0000","last_became_peered":"2024-09-15T01:44:07.707040+0000","last_unstale":"2024-09-15T01:44:10.574016+0000","last_undegraded":"2024-09-15T01:44:10.574016+0000","last_fullsized":"2024-09-15T01:44:10.574016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.796358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5079040,"num_objects":8,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":16,"num_read_kb":8,"num_write":20,"num_write_kb":4960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,8,10],"acting":[5,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,6,7,8,10,11],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.574289+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:44:09.574289+0000","last_peered":"2024-09-15T01:44:09.574289+0000","last_clean":"2024-09-15T01:44:09.574289+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:44:09.574289+0000","last_undegraded":"2024-09-15T01:44:09.574289+0000","last_fullsized":"2024-09-15T01:44:09.574289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'13","reported_seq":37,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.576409+0000","last_change":"2024-09-15T01:44:10.577281+0000","last_active":"2024-09-15T01:44:10.576326+0000","last_peered":"2024-09-15T01:44:09.808614+0000","last_clean":"2024-09-15T01:44:09.808614+0000","last_became_active":"2024-09-15T01:44:07.707503+0000","last_became_peered":"2024-09-15T01:44:07.707503+0000","last_unstale":"2024-09-15T01:44:10.576409+0000","last_undegraded":"2024-09-15T01:44:10.576409+0000","last_fullsized":"2024-09-15T01:44:10.576409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:33.459620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":32,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,5,10,2,6,0],"acting":[1,8,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,5,6,8,9,10,11],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.567505+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:44:09.567505+0000","last_peered":"2024-09-15T01:44:09.567505+0000","last_clean":"2024-09-15T01:44:09.567505+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:44:09.567505+0000","last_undegraded":"2024-09-15T01:44:09.567505+0000","last_fullsized":"2024-09-15T01:44:09.567505+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'8","reported_seq":30,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-15T01:44:10.584583+0000","last_change":"2024-09-15T01:44:10.584974+0000","last_active":"2024-09-15T01:44:10.584547+0000","last_peered":"2024-09-15T01:44:09.797104+0000","last_clean":"2024-09-15T01:44:09.797104+0000","last_became_active":"2024-09-15T01:44:07.578756+0000","last_became_peered":"2024-09-15T01:44:07.578756+0000","last_unstale":"2024-09-15T01:44:10.584583+0000","last_undegraded":"2024-09-15T01:44:10.584583+0000","last_fullsized":"2024-09-15T01:44:10.584583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.506200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":27,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2,4,7,9,10,11],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.573366+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:44:09.573366+0000","last_peered":"2024-09-15T01:44:09.573366+0000","last_clean":"2024-09-15T01:44:09.573366+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:44:09.573366+0000","last_undegraded":"2024-09-15T01:44:09.573366+0000","last_fullsized":"2024-09-15T01:44:09.573366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'9","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.779732+0000","last_change":"2024-09-15T01:44:07.580403+0000","last_active":"2024-09-15T01:44:09.779732+0000","last_peered":"2024-09-15T01:44:09.779732+0000","last_clean":"2024-09-15T01:44:09.779732+0000","last_became_active":"2024-09-15T01:44:07.578405+0000","last_became_peered":"2024-09-15T01:44:07.578405+0000","last_unstale":"2024-09-15T01:44:09.779732+0000","last_undegraded":"2024-09-15T01:44:09.779732+0000","last_fullsized":"2024-09-15T01:44:09.779732+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:31.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622016,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":9,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T01:44:09.569155+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:44:09.569155+0000","last_peered":"2024-09-15T01:44:09.569155+0000","last_clean":"2024-09-15T01:44:09.569155+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:44:09.569155+0000","last_undegraded":"2024-09-15T01:44:09.569155+0000","last_fullsized":"2024-09-15T01:44:09.569155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18989056,"num_objects":38,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":90,"num_read_kb":2200,"num_write":140,"num_write_kb":20592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2359296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45789184,"data_stored":45871104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":1,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411336,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302608,"kb_used_data":5252,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069232,"statfs":{"total":96636764160,"available":96326893568,"internally_reserved":0,"allocated":5378048,"data_stored":5269670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444038,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444039,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302200,"kb_used_data":4844,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069640,"statfs":{"total":96636764160,"available":96327311360,"internally_reserved":0,"allocated":4960256,"data_stored":4787386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301568,"kb_used_data":4212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070272,"statfs":{"total":96636764160,"available":96327958528,"internally_reserved":0,"allocated":4313088,"data_stored":4140218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]}]},{"osd":8,"up_from":17,"seq":73014444038,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299928,"kb_used_data":2572,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071912,"statfs":{"total":96636764160,"available":96329637888,"internally_reserved":0,"allocated":2633728,"data_stored":2460858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.837}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9770000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]}]},{"osd":6,"up_from":17,"seq":73014444039,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301484,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070356,"statfs":{"total":96636764160,"available":96328044544,"internally_reserved":0,"allocated":4227072,"data_stored":4054202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6120000000000001}]}]},{"osd":5,"up_from":17,"seq":73014444039,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301784,"kb_used_data":4428,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070056,"statfs":{"total":96636764160,"available":96327737344,"internally_reserved":0,"allocated":4534272,"data_stored":4357325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444038,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300552,"kb_used_data":3196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071288,"statfs":{"total":96636764160,"available":96328998912,"internally_reserved":0,"allocated":3272704,"data_stored":3099834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]},{"osd":3,"up_from":17,"seq":73014444038,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301376,"kb_used_data":4012,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070464,"statfs":{"total":96636764160,"available":96328155136,"internally_reserved":0,"allocated":4108288,"data_stored":3939081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1589999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.966}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444039,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300264,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071576,"statfs":{"total":96636764160,"available":96329293824,"internally_reserved":0,"allocated":2977792,"data_stored":2804922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]}]},{"osd":1,"up_from":17,"seq":73014444038,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302088,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069752,"statfs":{"total":96636764160,"available":96327426048,"internally_reserved":0,"allocated":4845568,"data_stored":4668621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":0,"up_from":17,"seq":73014444038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301184,"kb_used_data":3828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070656,"statfs":{"total":96636764160,"available":96328351744,"internally_reserved":0,"allocated":3919872,"data_stored":3750150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3596288,"data_stored":3596288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4513792,"data_stored":4513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2650112,"data_stored":2650112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3792896,"data_stored":3792896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2945024,"data_stored":2945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4202496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":3899392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4485120,"data_stored":4567040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":2306048,"data_stored":2306048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":3985408,"data_stored":3985408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4632576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:16.361 INFO:tasks.thrashosds.thrasher:Setting 3.e to [11, 9, 5, 4, 6, 10, 0, 1] 2024-09-15T01:44:16.361 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.e', '11', '9', '5', '4', '6', '10', '0', '1'] 2024-09-15T01:44:16.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.e 11 9 5 4 6 10 0 1 2024-09-15T01:44:16.364 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:16.364 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:16.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:16.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:16.387 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:16.411 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:16.433 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:16.436 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:16.432+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:16.455 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:16.513 INFO:tasks.rados.rados.0.smithi033.stdout:94: done (0 left) 2024-09-15T01:44:16.513 INFO:tasks.rados.rados.0.smithi033.stdout:95: read oid 3 snap -1 2024-09-15T01:44:16.513 INFO:tasks.rados.rados.0.smithi033.stdout:95: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:16.514 INFO:tasks.rados.rados.0.smithi033.stdout:96: snap_remove snap 0 2024-09-15T01:44:16.522 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:16.523 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:16.523 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:16.523 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:16.523 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:16.523 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:44:16.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:16.537 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:16.531+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:16.628 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.525684+0000", 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0981139960000004, 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0013215919999999999, 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.629 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525684+0000", 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525684+0000", 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.630 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525685+0000", 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.4200000000000005e-07 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525718+0000", 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3290999999999997e-05 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525735+0000", 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6218999999999998e-05 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.631 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525740+0000", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9259999999999997e-06 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525797+0000", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6462999999999998e-05 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.525805+0000", 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.4689999999999997e-06 2024-09-15T01:44:16.632 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.526919+0000", 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0011133899999999999 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527006+0000", 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6991999999999997e-05 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.633 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.9s7 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.527203+0000", 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0965953979999998, 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0023617439999999998, 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527203+0000", 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.634 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527203+0000", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527203+0000", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1299999999999999e-07 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527511+0000", 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00030723799999999999 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.635 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527531+0000", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0732000000000001e-05 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527539+0000", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6790000000000005e-06 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527595+0000", 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.5680999999999998e-05 2024-09-15T01:44:16.636 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.527611+0000", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6477000000000002e-05 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.529536+0000", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0019247509999999999 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.529565+0000", 2024-09-15T01:44:16.637 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8473e-05 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.9s7 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.532534+0000", 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.091264035, 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.002738814, 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.638 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532534+0000", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532534+0000", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532535+0000", 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9699999999999995e-07 2024-09-15T01:44:16.639 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532608+0000", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2837999999999996e-05 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532626+0000", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7955e-05 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532634+0000", 2024-09-15T01:44:16.640 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.4069999999999993e-06 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532726+0000", 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1421000000000004e-05 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.532737+0000", 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1918e-05 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.641 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535251+0000", 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0025137470000000002 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.642 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535273+0000", 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.1730999999999999e-05 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.3s7 33/32 ECSubRead(tid=8, to_read={3:c5b7678e:::smithi03327189-28:head=0,114688,0}, subchunks={3:c5b7678e:::smithi03327189-28:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.533358+0000", 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0904402129999999, 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0013523490000000001, 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.643 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533358+0000", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533358+0000", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.644 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533359+0000", 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533367+0000", 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0819999999999999e-06 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533379+0000", 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2081e-05 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.645 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533403+0000", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.4026999999999999e-05 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533459+0000", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6777000000000002e-05 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.534710+0000", 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001250805 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.646 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.3s7 33/32 ECSubRead(tid=9, to_read={3:c5b7678e:::smithi03327189-28:head=0,114688,0}, subchunks={3:c5b7678e:::smithi03327189-28:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.533421+0000", 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0903774510000002, 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0014422269999999999, 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533421+0000", 2024-09-15T01:44:16.647 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533421+0000", 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533421+0000", 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8999999999999998e-07 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.648 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533426+0000", 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.5519999999999998e-06 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533432+0000", 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.798e-06 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533451+0000", 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8766e-05 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.649 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.534748+0000", 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001296866 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.534863+0000", 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000114955 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.650 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.3s7 33/32 ECSubRead(tid=10, to_read={3:c5b7678e:::smithi03327189-28:head=0,114688,0}, subchunks={3:c5b7678e:::smithi03327189-28:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.533474+0000", 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0903237499999996, 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001565653, 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533474+0000", 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.651 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533474+0000", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533475+0000", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.34e-07 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533482+0000", 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.9129999999999997e-06 2024-09-15T01:44:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533491+0000", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.6260000000000007e-06 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.533497+0000", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.0270000000000001e-06 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.534918+0000", 2024-09-15T01:44:16.653 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001420957 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535040+0000", 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000121796 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.535910+0000", 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0878884930000003, 2024-09-15T01:44:16.654 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001533027, 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535910+0000", 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535910+0000", 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.655 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535910+0000", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6000000000000003e-07 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535929+0000", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8329e-05 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535945+0000", 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6062999999999999e-05 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.656 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.535953+0000", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.7770000000000002e-06 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536012+0000", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9596000000000003e-05 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536026+0000", 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3909e-05 2024-09-15T01:44:16.657 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537414+0000", 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001388161 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537443+0000", 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8432000000000002e-05 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.658 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.9s7 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.536707+0000", 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0870914970000003, 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032926710000000001, 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536707+0000", 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.659 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536707+0000", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536707+0000", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1800000000000005e-07 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536728+0000", 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0500999999999999e-05 2024-09-15T01:44:16.660 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536746+0000", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8153000000000001e-05 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536753+0000", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3919999999999997e-06 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536804+0000", 2024-09-15T01:44:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.0507000000000003e-05 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.536814+0000", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0409e-05 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.539949+0000", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003134623 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.539999+0000", 2024-09-15T01:44:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.0368000000000003e-05 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/32 ECSubWrite(tid=29, reqid=client.4632.0:210, at_version=33'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.543678+0000", 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0801205930000002, 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032753779999999998, 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.543678+0000", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.543678+0000", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.543678+0000", 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.0999999999999998e-07 2024-09-15T01:44:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545343+0000", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001664614 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545359+0000", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5648999999999999e-05 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545367+0000", 2024-09-15T01:44:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.303e-06 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545420+0000", 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.2697e-05 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545431+0000", 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1267000000000001e-05 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.666 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.546931+0000", 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0014999620000000001 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.546953+0000", 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2175999999999999e-05 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.9s7 33/32 ECSubWrite(tid=5, reqid=client.4632.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.544752+0000", 2024-09-15T01:44:16.667 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0790465569999999, 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00063352299999999999, 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544752+0000", 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544752+0000", 2024-09-15T01:44:16.668 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544753+0000", 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1299999999999998e-07 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544772+0000", 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9327999999999999e-05 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.669 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544791+0000", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8827000000000001e-05 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544799+0000", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9270000000000005e-06 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544859+0000", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.0676e-05 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.670 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.544873+0000", 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3642000000000001e-05 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545351+0000", 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00047767699999999997 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545385+0000", 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.4533000000000003e-05 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.671 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/32 ECSubWrite(tid=30, reqid=client.4632.0:211, at_version=33'9, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.548306+0000", 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0754925609999999, 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0021851919999999999, 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548306+0000", 2024-09-15T01:44:16.672 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548306+0000", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548306+0000", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3300000000000002e-07 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548441+0000", 2024-09-15T01:44:16.673 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000134795 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548460+0000", 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8972000000000001e-05 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548467+0000", 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.674 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548527+0000", 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9604000000000001e-05 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548539+0000", 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2516e-05 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550462+0000", 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001922918 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.675 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550491+0000", 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8399999999999999e-05 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.cs3 33/32 ECSubWrite(tid=66, reqid=client.4632.0:197, at_version=33'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.549289+0000", 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0745096040000002, 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0050872010000000004, 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.676 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.549289+0000", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.549289+0000", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.549289+0000", 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.5899999999999995e-07 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00069517899999999998 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550013+0000", 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8846000000000001e-05 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.678 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550023+0000", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.9170000000000001e-06 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550091+0000", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.7362999999999999e-05 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.550110+0000", 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9772e-05 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.679 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554331+0000", 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0042203529999999996 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554376+0000", 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.4912e-05 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.680 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/32 ECSubWrite(tid=31, reqid=client.4632.0:212, at_version=33'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.552061+0000", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0717366769999996, 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027119990000000001, 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552061+0000", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.681 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552061+0000", 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552062+0000", 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1099999999999995e-07 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552082+0000", 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0115e-05 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.682 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552097+0000", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4276e-05 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552104+0000", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2300000000000002e-06 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552169+0000", 2024-09-15T01:44:16.683 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5628000000000003e-05 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.552186+0000", 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6385e-05 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554736+0000", 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0025500169999999999 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.684 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554773+0000", 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.7636999999999999e-05 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.cs3 33/32 ECSubWrite(tid=69, reqid=client.4632.0:197, at_version=0'0, trim_to=0'0, roll_forward_to=33'12))", 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.558909+0000", 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0648892339999998, 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00099410699999999993, 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.685 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558909+0000", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558909+0000", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558910+0000", 2024-09-15T01:44:16.686 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2099999999999995e-07 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558931+0000", 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0662999999999999e-05 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558948+0000", 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7516999999999998e-05 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.687 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.558959+0000", 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0562e-05 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.559032+0000", 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3744000000000004e-05 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.559050+0000", 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7408999999999999e-05 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.688 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.559870+0000", 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00082010999999999998 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.559903+0000", 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3181e-05 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.689 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/32 ECSubWrite(tid=35, reqid=client.4632.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=33'10))", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.560903+0000", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0628954530000003, 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0027913159999999998, 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560903+0000", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.690 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560903+0000", 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560904+0000", 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2500000000000004e-07 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560927+0000", 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.3844000000000001e-05 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.691 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560947+0000", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9504000000000002e-05 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.560956+0000", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4010000000000003e-06 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.561049+0000", 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2713000000000005e-05 2024-09-15T01:44:16.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.561063+0000", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4283e-05 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.563652+0000", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0025885040000000002 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.563694+0000", 2024-09-15T01:44:16.693 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2141999999999997e-05 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.1s2 33/32 ECSubRead(tid=39, to_read={3:85bd41e4:::smithi03327189-14:3=0,262144,0}, subchunks={3:85bd41e4:::smithi03327189-14:3=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.585891+0000", 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.037907315, 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00065136399999999998, 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.694 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585891+0000", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585891+0000", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585892+0000", 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8800000000000001e-07 2024-09-15T01:44:16.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585902+0000", 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.9159999999999996e-06 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585920+0000", 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8681e-05 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.696 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.585928+0000", 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.8320000000000006e-06 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.586022+0000", 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3938999999999996e-05 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.586542+0000", 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00052010800000000003 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.697 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.1s2 33/32 ECSubRead(tid=40, to_read={3:85bd41e4:::smithi03327189-14:head=0,262144,0}, subchunks={3:85bd41e4:::smithi03327189-14:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.590204+0000", 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0335943539999999, 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00094632799999999997, 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590204+0000", 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.698 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590204+0000", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590205+0000", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.5600000000000005e-07 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590212+0000", 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1380000000000001e-06 2024-09-15T01:44:16.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590226+0000", 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3978999999999999e-05 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590232+0000", 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.7089999999999996e-06 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.700 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.590295+0000", 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.2762999999999995e-05 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.591150+0000", 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00085498300000000002 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/32 ECSubWrite(tid=81, reqid=client.4632.0:200, at_version=33'23, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.608811+0000", 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0149873850000004, 2024-09-15T01:44:16.701 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0013585240000000001, 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608811+0000", 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608811+0000", 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.702 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608812+0000", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.093e-06 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608850+0000", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.7803999999999999e-05 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608874+0000", 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.4592000000000001e-05 2024-09-15T01:44:16.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608889+0000", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5095e-05 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608974+0000", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.4640000000000003e-05 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.608987+0000", 2024-09-15T01:44:16.704 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3038e-05 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.610127+0000", 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001140077 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.610169+0000", 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2184999999999997e-05 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.705 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/32 ECSubWrite(tid=82, reqid=client.4632.0:201, at_version=33'24, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.611350+0000", 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0124480939999998, 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001067583, 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611350+0000", 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.706 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611350+0000", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611351+0000", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.299e-06 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611391+0000", 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.9303e-05 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.707 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611416+0000", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5137999999999999e-05 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611425+0000", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4609999999999997e-06 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611526+0000", 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00010042 2024-09-15T01:44:16.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.611543+0000", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7479e-05 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.612381+0000", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00083772600000000001 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.612418+0000", 2024-09-15T01:44:16.709 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.6757000000000001e-05 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/32 ECSubWrite(tid=83, reqid=client.4632.0:201, at_version=0'0, trim_to=0'0, roll_forward_to=33'24))", 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.615516+0000", 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.0082824370000001, 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.001226715, 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.710 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615516+0000", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615516+0000", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615517+0000", 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.474e-06 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.711 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615545+0000", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.762e-05 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615576+0000", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1479000000000001e-05 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615591+0000", 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4714e-05 2024-09-15T01:44:16.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615702+0000", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00011142299999999999 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_started", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.615722+0000", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9660000000000002e-05 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "sub_op_committed", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.616697+0000", 2024-09-15T01:44:16.713 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00097515199999999996 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.616742+0000", 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.5192999999999999e-05 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:16.714 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:16.716 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:16.716 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:16.708+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:16.816 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:16.811+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:16.821 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:16.821 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:16.821 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:16.822 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:16.831 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:16.917 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:16.911+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:16.923 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:16.923 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:16.923 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:16.923 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:16.924 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:16.932 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:17.018 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:17.011+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:17.081 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.635367853999998, 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.082 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:17.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.085 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:17.086 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.634406327000001, 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.087 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:17.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:17.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:17.090 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.631841087000002, 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:17.091 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.092 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:17.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.095 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.630879594, 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.096 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:17.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:17.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.628797035000002, 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.100 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:17.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:17.102 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.103 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:17.104 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.627765698000001, 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:17.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.107 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.108 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.626835149000001, 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.111 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.112 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.625979740999998, 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:17.113 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.114 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.115 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.116 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.117 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.623463319999999, 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.118 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:17.119 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.120 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.071423608, 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.122 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:17.123 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.125 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.126 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.070009383999999, 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.127 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.128 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.130 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.069003362, 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:17.131 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.132 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.134 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 22.067732734, 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.135 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.136 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.137 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:17.138 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.139 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 21.512034654000001, 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:17.140 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.142 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:17.143 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.144 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=3, to_read={3:720ccbad:::smithi03327189-29:head=0,262144,0}, subchunks={3:720ccbad:::smithi03327189-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5402768120000001, 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033057709999999999, 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.145 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537073+0000", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2750000000000001e-06 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537091+0000", 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537118+0000", 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7220999999999999e-05 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537130+0000", 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1551e-05 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537215+0000", 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.5156999999999998e-05 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.540378+0000", 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0031629000000000002 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.148 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=7, to_read={3:7de15fb4:::smithi03327189-49:head=0,262144,0}, subchunks={3:7de15fb4:::smithi03327189-49:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5320433930000004, 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033061729999999999, 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.149 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545307+0000", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0699999999999999e-06 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545320+0000", 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2867000000000001e-05 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.150 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545339+0000", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9352e-05 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8869999999999998e-06 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545406+0000", 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7782999999999999e-05 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.151 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548612+0000", 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032062140000000002 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.7s1 33/32 ECSubRead(tid=46, to_read={3:e02ccbe4:::smithi03327189-36:head=0,262144,0}, subchunks={3:e02ccbe4:::smithi03327189-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5261769420000002, 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033818339999999998, 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.152 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.153 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551173+0000", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.111e-06 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551185+0000", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.205e-05 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551202+0000", 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6971999999999999e-05 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.154 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551209+0000", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5339999999999996e-06 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551283+0000", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.4664000000000002e-05 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554554+0000", 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032705030000000001 2024-09-15T01:44:17.155 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=67, to_read={3:ac2d3331:::smithi03327189-18:head=0,262144,0}, subchunks={3:ac2d3331:::smithi03327189-18:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5242251979999999, 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003506547, 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.156 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553125+0000", 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2900000000000002e-07 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.157 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553136+0000", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1032999999999999e-05 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553161+0000", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5043000000000001e-05 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553170+0000", 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8729999999999996e-06 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.158 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553235+0000", 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5230999999999999e-05 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556630+0000", 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003395538 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.159 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.cs7 33/32 ECSubRead(tid=68, to_read={3:3584cd82:::smithi03327189-23:head=0,262144,0}, subchunks={3:3584cd82:::smithi03327189-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5241617510000003, 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036502639999999999, 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:17.160 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553188+0000", 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2599999999999998e-07 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553194+0000", 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3770000000000001e-06 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.161 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553204+0000", 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553210+0000", 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.8850000000000004e-06 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553352+0000", 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014202600000000001 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.162 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556838+0000", 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003485868 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=71, to_read={3:a22eae17:::smithi03327189-26:head=0,262144,0}, subchunks={3:a22eae17:::smithi03327189-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 4.5118586609999998, 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0030758130000000002, 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.163 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565491+0000", 2024-09-15T01:44:17.164 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2699999999999998e-07 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565507+0000", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6076e-05 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565523+0000", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565529+0000", 2024-09-15T01:44:17.165 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565605+0000", 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6344e-05 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.568566+0000", 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00296131 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.166 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:17.167 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:17.167 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:17.168 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:17.169 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:17.164+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:17.269 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:17.263+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:17.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:17.370 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:17.364+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:17.392 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:17.393 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:17.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:17.471 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:17.468+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9682028520000001, 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:17.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:17.504 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9657714540000004, 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:17.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9655326390000001, 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:17.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:17.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9655012559999996, 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.518 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:17.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:17.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9862938259999998, 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:17.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:17.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.528 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9783794239999999, 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:17.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9783501609999998, 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:17.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9509133150000002, 2024-09-15T01:44:17.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9497465700000003, 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9493732970000002, 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9486541989999999, 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9471650929999997, 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00703289, 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547486+0000", 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:44:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547490+0000", 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1160000000000001e-06 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547502+0000", 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2284e-05 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547509+0000", 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8709999999999999e-06 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554232+0000", 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067227739999999999 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554518+0000", 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028611400000000002 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9471018100000004, 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006755016, 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547552+0000", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.875e-06 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547560+0000", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2099999999999993e-06 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547565+0000", 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.843e-06 2024-09-15T01:44:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554285+0000", 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067204109999999999 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554304+0000", 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8223000000000002e-05 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9470644969999995, 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074053929999999997, 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:17.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547589+0000", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7970000000000001e-06 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547597+0000", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7959999999999996e-06 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547601+0000", 2024-09-15T01:44:17.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4259999999999996e-06 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554329+0000", 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067279790000000003 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554991+0000", 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000661928 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=30, last_complete=33'9, committed=1, applied=1))", 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9461417069999998, 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070855279999999998, 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548510+0000", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3689999999999999e-06 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548515+0000", 2024-09-15T01:44:17.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.259e-06 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548532+0000", 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6586999999999999e-05 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548540+0000", 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7670000000000003e-06 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555036+0000", 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006496155 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555594+0000", 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00055839099999999999 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=30, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9451856029999997, 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061816969999999999, 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549465+0000", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549466+0000", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2999999999999999e-07 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549471+0000", 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.344e-06 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549483+0000", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1627000000000001e-05 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9030000000000001e-06 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555629+0000", 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061406680000000002 2024-09-15T01:44:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555646+0000", 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7325000000000001e-05 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9385509580000004, 2024-09-15T01:44:17.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067647840000000002, 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556100+0000", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4499999999999998e-07 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556180+0000", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9863000000000004e-05 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556188+0000", 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:44:17.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556193+0000", 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7709999999999997e-06 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562617+0000", 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064236800000000002 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562864+0000", 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000247366 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9385195550000001, 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064187080000000004, 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556131+0000", 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556155+0000", 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4170999999999999e-05 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556162+0000", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0190000000000001e-06 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561747+0000", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055845870000000002 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562549+0000", 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000802931 2024-09-15T01:44:17.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9380170190000001, 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066845059999999998, 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556653+0000", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0038999999999999e-05 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556659+0000", 2024-09-15T01:44:17.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9510000000000003e-06 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562747+0000", 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060879640000000004 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563318+0000", 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00057055199999999999 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9377748199999996, 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007571698, 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556895+0000", 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9117e-05 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556900+0000", 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2440000000000001e-06 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563373+0000", 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064735360000000002 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564447+0000", 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010738010000000001 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.589 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:17.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:17.591 INFO:tasks.rados.rados.0.smithi033.stdout:96: done (1 left) 2024-09-15T01:44:17.592 INFO:tasks.rados.rados.0.smithi033.stdout:97: snap_remove snap 4 2024-09-15T01:44:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:set 3.e pg_upmap mapping to [11,9,5,4,6,10,0,1] 2024-09-15T01:44:17.593 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:17.584+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:17.692 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:17.688+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:17.743 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.743 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:17.743 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:17.743 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:17.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:17.793 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:17.788+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:17.855 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:17.856 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:17.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:17.894 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:17.891+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4439808459999997, 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:17.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.443941841, 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4439108970000003, 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:17.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:17.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:17.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4384648909999997, 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004275114, 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526502+0000", 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526503+0000", 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2900000000000002e-07 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526853+0000", 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00035008499999999998 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526864+0000", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1246e-05 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526871+0000", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8299999999999998e-06 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526941+0000", 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0291999999999997e-05 2024-09-15T01:44:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526954+0000", 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2208000000000001e-05 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530752+0000", 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037984220000000001 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530777+0000", 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5102000000000001e-05 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4330010140000002, 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0057431050000000001, 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531967+0000", 2024-09-15T01:44:17.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000233353 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2055999999999999e-05 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.635e-06 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532530+0000", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031128500000000001 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532541+0000", 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0937999999999999e-05 2024-09-15T01:44:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537675+0000", 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051336020000000001 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537709+0000", 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4424000000000001e-05 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=62, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4305242360000001, 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055656810000000003, 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534451+0000", 2024-09-15T01:44:17.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5809999999999999e-06 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534467+0000", 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6405e-05 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534475+0000", 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0840000000000003e-06 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534924+0000", 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044986499999999998 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540008+0000", 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050840019999999998 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.997 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=63, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4304320869999998, 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058134440000000001, 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:17.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.13e-07 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534542+0000", 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3890000000000002e-06 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534557+0000", 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4919000000000001e-05 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534562+0000", 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7189999999999998e-06 2024-09-15T01:44:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540154+0000", 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055920869999999999 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540348+0000", 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019381699999999999 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=64, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4303988009999999, 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059416290000000004, 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534573+0000", 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534579+0000", 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6479999999999997e-06 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534584+0000", 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6079999999999998e-06 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540397+0000", 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058133880000000001 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540510+0000", 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001125 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4295197780000004, 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:18.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4282548979999996, 2024-09-15T01:44:18.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041052659999999998, 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536712+0000", 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536713+0000", 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8999999999999996e-07 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536737+0000", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3980000000000001e-05 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536757+0000", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0792e-05 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536767+0000", 2024-09-15T01:44:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4409999999999999e-06 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536814+0000", 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7348000000000001e-05 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536825+0000", 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0723999999999999e-05 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540785+0000", 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039597180000000001 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540817+0000", 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2573000000000001e-05 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.449026597, 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:18.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:18.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/34 ECSubWrite(tid=56, reqid=osd.3.0:56, at_version=37'14, trim_to=0'0, roll_forward_to=35'10))", 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.43719865, 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049829160000000004, 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527769+0000", 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:44:18.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527795+0000", 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5474e-05 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527809+0000", 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4041e-05 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527816+0000", 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527865+0000", 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9304000000000001e-05 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527876+0000", 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.049e-05 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532721+0000", 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004845678 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532751+0000", 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9935000000000001e-05 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=93, reqid=osd.5.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.528459+0000", 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.4365080190000001, 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042753859999999999, 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528459+0000", 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528459+0000", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528460+0000", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9100000000000003e-07 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528472+0000", 2024-09-15T01:44:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2673999999999999e-05 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528490+0000", 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8005000000000001e-05 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528498+0000", 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5009999999999998e-06 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528548+0000", 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9901999999999998e-05 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528564+0000", 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6033999999999999e-05 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532702+0000", 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041386510000000001 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532734+0000", 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1828000000000001e-05 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.44268516800000002, 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045657379999999997, 2024-09-15T01:44:18.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522283+0000", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-07 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522313+0000", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0176999999999999e-05 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522329+0000", 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5685999999999999e-05 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522338+0000", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1230000000000006e-06 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522428+0000", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0483999999999996e-05 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522441+0000", 2024-09-15T01:44:18.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3192e-05 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526819+0000", 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043777 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526847+0000", 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8388999999999999e-05 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 38/34 ECSubWrite(tid=58, reqid=osd.3.0:58, at_version=38'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.528905+0000", 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.43606214199999999, 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039810590000000003, 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528905+0000", 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528905+0000", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528905+0000", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2399999999999999e-07 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528932+0000", 2024-09-15T01:44:18.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6998000000000001e-05 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528943+0000", 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0709e-05 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528949+0000", 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8980000000000001e-06 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528995+0000", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6424999999999997e-05 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529005+0000", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0086e-05 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532862+0000", 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00385666 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532886+0000", 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4059000000000001e-05 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=59, reqid=osd.3.0:59, at_version=38'16, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.43558182000000001, 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062207900000000003, 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529386+0000", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1200000000000002e-07 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529410+0000", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4343000000000001e-05 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529422+0000", 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2128e-05 2024-09-15T01:44:18.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529429+0000", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7750000000000004e-06 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529473+0000", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3523000000000003e-05 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529479+0000", 2024-09-15T01:44:18.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3720000000000001e-06 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535566+0000", 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006087528 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535606+0000", 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9409000000000002e-05 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s7 38/34 ECSubWrite(tid=43, reqid=osd.0.0:42, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.43366830499999998, 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004403382, 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8000000000000004e-07 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531312+0000", 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2727e-05 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531340+0000", 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8240000000000001e-05 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531348+0000", 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4270000000000002e-06 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531402+0000", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4184000000000002e-05 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531412+0000", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6970000000000005e-06 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535669+0000", 2024-09-15T01:44:18.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042573929999999999 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535702+0000", 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2833999999999998e-05 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 38/34 ECSubWrite(tid=98, reqid=osd.5.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.531399+0000", 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.433567761, 2024-09-15T01:44:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004203264, 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531399+0000", 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531399+0000", 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531400+0000", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5700000000000002e-07 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531409+0000", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0480000000000004e-06 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531418+0000", 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.302e-06 2024-09-15T01:44:18.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531424+0000", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8590000000000001e-06 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531469+0000", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4910000000000002e-05 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531478+0000", 2024-09-15T01:44:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7860000000000002e-06 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535575+0000", 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040977239999999996 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535602+0000", 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6978000000000002e-05 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=60, reqid=osd.3.0:60, at_version=38'18, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.432805408, 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060140539999999996, 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:18.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8100000000000001e-07 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532195+0000", 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2177e-05 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532208+0000", 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2991e-05 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532217+0000", 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8309999999999996e-06 2024-09-15T01:44:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532269+0000", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080000000000003e-05 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532279+0000", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6260000000000007e-06 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538153+0000", 2024-09-15T01:44:18.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058734499999999997 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538176+0000", 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3017999999999999e-05 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 38/34 ECSubWrite(tid=61, reqid=osd.3.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=38'8))", 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:18.056 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.43158073600000002, 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048073339999999999, 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:18.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533387+0000", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.06e-07 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533396+0000", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533405+0000", 2024-09-15T01:44:18.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9239999999999996e-06 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533409+0000", 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6630000000000001e-06 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533473+0000", 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3547999999999995e-05 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533497+0000", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3768999999999999e-05 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538175+0000", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046780399999999996 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538194+0000", 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8910999999999999e-05 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.061 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.061 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:18.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:18.063 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:18.056+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:18.163 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:18.160+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:18.168 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:18.169 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:18.169 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:18.169 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:18.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:18.265 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:18.260+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:18.277 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:18.278 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:18.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:18.365 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:18.360+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:18.388 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:18.388 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:18.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:18.388 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4317.0:1 2.3 e21/20 2:c4c92e5a:::rbd_trash:head v 21'1, mlcod=0'0)", 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.986141980999999, 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087692900000000003, 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397126+0000", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2399999999999999e-07 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397131+0000", 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9860000000000002e-06 2024-09-15T01:44:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397145+0000", 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4652e-05 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397160+0000", 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4124000000000001e-05 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397218+0000", 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7942999999999997e-05 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397265+0000", 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6953999999999998e-05 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.397961+0000", 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00069677499999999996 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:43:55.398002+0000", 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1171000000000003e-05 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8572461110000003, 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003107659, 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526022+0000", 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526051+0000", 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8187e-05 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526061+0000", 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0343000000000001e-05 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526068+0000", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7469999999999999e-06 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526304+0000", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000236197 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526315+0000", 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0946e-05 2024-09-15T01:44:18.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529101+0000", 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027858359999999999 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529129+0000", 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8313000000000001e-05 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8477753620000001, 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:18.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.es0 33/32 [PushOp(3:720ccbad:::smithi03327189-29:head, version: 30'2, data_included: [0~118784], data_size: 118784, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:720ccbad:::smithi03327189-29:head@30'2, size: 475136, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:475136, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8425106009999999, 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00112323, 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540757+0000", 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541186+0000", 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042845600000000002 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541204+0000", 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7867999999999999e-05 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541215+0000", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1609e-05 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541347+0000", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000132008 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.541880+0000", 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000532962 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.405 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/32 ECSubWrite(tid=66, reqid=client.4632.0:197, at_version=33'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.834509401, 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026146149999999998, 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548758+0000", 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548760+0000", 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.663e-06 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549278+0000", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051834099999999996 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549296+0000", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7742000000000001e-05 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549304+0000", 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3620000000000006e-06 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549384+0000", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0055999999999994e-05 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549396+0000", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1759999999999999e-05 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551342+0000", 2024-09-15T01:44:18.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001946138 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551373+0000", 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0552999999999997e-05 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.es0 33/32 [PushOp(3:7de15fb4:::smithi03327189-49:head, version: 30'5, data_included: [0~204800], data_size: 204800, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:7de15fb4:::smithi03327189-49:head@30'5, size: 819200, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:819200, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8337790429999998, 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00108639, 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549488+0000", 2024-09-15T01:44:18.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549489+0000", 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.23e-07 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549872+0000", 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038235 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549900+0000", 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8373000000000001e-05 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549913+0000", 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3258000000000001e-05 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550006+0000", 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2798000000000002e-05 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550575+0000", 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056878799999999998 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.7s1 33/32 [PushOp(3:e02ccbe4:::smithi03327189-36:head, version: 30'8, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:e02ccbe4:::smithi03327189-36:head@30'8, size: 1490944, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8275963940000004, 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010742620000000001, 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555671+0000", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555672+0000", 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.540000000000001e-07 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555831+0000", 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000158868 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555856+0000", 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4610999999999999e-05 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555868+0000", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2492e-05 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555951+0000", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2587000000000001e-05 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556745+0000", 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079474999999999999 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDPGPush(3.3s4 33/32 [PushOp(3:c5b7678e:::smithi03327189-28:head, version: 30'4, data_included: [0~114688], data_size: 114688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:c5b7678e:::smithi03327189-28:head@30'4, size: 458752, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:458752, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8250790759999997, 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001032888, 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558188+0000", 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558189+0000", 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3399999999999998e-07 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558666+0000", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00047639900000000002 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558683+0000", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7101000000000001e-05 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558692+0000", 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5559999999999996e-06 2024-09-15T01:44:18.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558781+0000", 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9085999999999994e-05 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559221+0000", 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044001199999999999 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.421 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/32 ECSubWrite(tid=69, reqid=client.4632.0:197, at_version=0'0, trim_to=0'0, roll_forward_to=33'12))", 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8247596230000003, 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00098901999999999996, 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558508+0000", 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558509+0000", 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1999999999999999e-07 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558529+0000", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0579000000000001e-05 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558557+0000", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7719999999999999e-05 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558572+0000", 2024-09-15T01:44:18.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5607000000000002e-05 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558627+0000", 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5025000000000003e-05 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.558639+0000", 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1578e-05 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559471+0000", 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00083245300000000003 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559497+0000", 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5437999999999999e-05 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.cs5 33/32 ECSubRead(tid=74, to_read={3:3cf1f308:::smithi03327189-35:head=0,262144,0}, subchunks={3:3cf1f308:::smithi03327189-35:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8092738710000003, 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010707749999999999, 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573994+0000", 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.573995+0000", 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2810000000000001e-06 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574023+0000", 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7721000000000001e-05 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574065+0000", 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1881000000000001e-05 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574074+0000", 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7620000000000007e-06 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.574152+0000", 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7248000000000003e-05 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.575064+0000", 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091288199999999995 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=81, reqid=client.4632.0:200, at_version=33'23, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.774990045, 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001931767, 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608277+0000", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608279+0000", 2024-09-15T01:44:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5969999999999999e-06 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608325+0000", 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5448000000000003e-05 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608364+0000", 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9894000000000001e-05 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608379+0000", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4249e-05 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608512+0000", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013341699999999999 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.608534+0000", 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2113e-05 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610148+0000", 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016133300000000001 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610209+0000", 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1718999999999996e-05 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=82, reqid=client.4632.0:201, at_version=33'24, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.772454872, 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001354173, 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610813+0000", 2024-09-15T01:44:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610814+0000", 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3200000000000001e-06 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610857+0000", 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3056000000000002e-05 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610886+0000", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8645000000000001e-05 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610899+0000", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3173000000000001e-05 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.610982+0000", 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3129999999999999e-05 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.611003+0000", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.126e-05 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.612129+0000", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011261750000000001 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.612167+0000", 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7413999999999998e-05 2024-09-15T01:44:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=83, reqid=client.4632.0:201, at_version=0'0, trim_to=0'0, roll_forward_to=33'24))", 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.7682668599999998, 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001080248, 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:18.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615001+0000", 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0200000000000001e-07 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615035+0000", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3966e-05 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615052+0000", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615060+0000", 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2740000000000002e-06 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615145+0000", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4999000000000004e-05 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.615168+0000", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3722000000000001e-05 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.616041+0000", 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087272599999999999 2024-09-15T01:44:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.616081+0000", 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9888999999999997e-05 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.8673562860000001, 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040598090000000002, 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515912+0000", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0800000000000005e-07 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515965+0000", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3362999999999997e-05 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515984+0000", 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9130000000000001e-05 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516042+0000", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7518000000000001e-05 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516184+0000", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014263599999999999 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516230+0000", 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5240000000000001e-05 2024-09-15T01:44:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519914+0000", 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036847939999999999 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519971+0000", 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6619999999999997e-05 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/34 ECSubWrite(tid=93, reqid=osd.5.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.528478+0000", 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.854789113, 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024435659999999999, 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528478+0000", 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528478+0000", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528480+0000", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.251e-06 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528495+0000", 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5809000000000001e-05 2024-09-15T01:44:18.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528516+0000", 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0404e-05 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528525+0000", 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4020000000000008e-06 2024-09-15T01:44:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528622+0000", 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7093e-05 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.528635+0000", 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2598e-05 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.530884+0000", 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002249366 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.530922+0000", 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7642999999999999e-05 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.452 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.86401039199999996, 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:18.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.86385946400000002, 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:18.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:18.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.86125702599999998, 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050197059999999996, 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522012+0000", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.122e-06 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522045+0000", 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3034999999999998e-05 2024-09-15T01:44:18.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522067+0000", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2742e-05 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522078+0000", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0278000000000001e-05 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524135+0000", 2024-09-15T01:44:18.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020572450000000001 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524153+0000", 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8510000000000001e-05 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527017+0000", 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028639719999999998 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527030+0000", 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2802e-05 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=97, reqid=osd.5.0:95, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.85454283099999995, 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041007480000000004, 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528726+0000", 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8700000000000004e-07 2024-09-15T01:44:18.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528739+0000", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3694e-05 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528757+0000", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501000000000001e-05 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528763+0000", 2024-09-15T01:44:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7270000000000001e-06 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528985+0000", 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000221955 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528996+0000", 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0363000000000001e-05 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532796+0000", 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003800257 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532825+0000", 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9363999999999999e-05 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=98, reqid=osd.5.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.538432+0000", 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.84483556500000001, 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027629389999999998, 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538432+0000", 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538432+0000", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538433+0000", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0359999999999999e-06 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538449+0000", 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5807999999999998e-05 2024-09-15T01:44:18.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538463+0000", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4151999999999999e-05 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538469+0000", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7780000000000004e-06 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538520+0000", 2024-09-15T01:44:18.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1242999999999997e-05 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538531+0000", 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1396999999999999e-05 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.541174+0000", 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026424209999999998 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:18.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.541195+0000", 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1104e-05 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:18.475 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:18.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:18.477 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:18.468+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:18.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:18.520 INFO:tasks.rados.rados.0.smithi033.stdout:97: done (1 left) 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:98: rollback oid 43 current snap is 5 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 43 to 3 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:99: read oid 38 snap -1 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:99: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:100: delete oid 49 current snap is 5 2024-09-15T01:44:18.521 INFO:tasks.rados.rados.0.smithi033.stdout:95: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:18.536 INFO:tasks.rados.rados.0.smithi033.stdout:99: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-15T01:44:18.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:44:18.550 INFO:tasks.rados.rados.0.smithi033.stdout:98: finishing rollback tid 0 to smithi03327189-43 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:98: finishing rollback tid 1 to smithi03327189-43 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:95: done (3 left) 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:99: done (2 left) 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:100: done (1 left) 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:101: rmattr oid 1 current snap is 5 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:101: done (1 left) 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:102: delete oid 11 current snap is 5 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:98: finishing rollback tid 2 to smithi03327189-43 2024-09-15T01:44:18.551 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T01:44:18.556 INFO:tasks.rados.rados.0.smithi033.stdout:98: done (1 left) 2024-09-15T01:44:18.556 INFO:tasks.rados.rados.0.smithi033.stdout:102: done (0 left) 2024-09-15T01:44:18.556 INFO:tasks.rados.rados.0.smithi033.stdout:103: snap_remove snap 3 2024-09-15T01:44:18.577 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:18.572+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:18.622 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:18.632 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:18.632 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:18.643 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:18.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:18.664 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:18.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:18.683 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:18.676+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:18.704 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:18.728 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:18.749 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:18.754 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:18.754 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:18.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:18.771 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:18.778 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:18.771+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:18.796 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:18.820 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:18.842 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:18.865 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:18.865 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:18.865 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:18.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:18.924 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:18.920+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:18.981 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:18.975+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:18.988 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:18.988 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:18.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:19.081 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:19.075+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:19.095 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.095 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:19.182 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:19.176+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:19.209 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.209 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:19.283 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:19.275+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:19.320 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.320 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:19.384 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:19.376+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:19.431 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.431 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:19.484 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:19.480+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:19.527 INFO:tasks.rados.rados.0.smithi033.stdout:103: done (0 left) 2024-09-15T01:44:19.527 INFO:tasks.rados.rados.0.smithi033.stdout:104: delete oid 22 current snap is 5 2024-09-15T01:44:19.536 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:44:19.540 INFO:tasks.rados.rados.0.smithi033.stdout:104: done (0 left) 2024-09-15T01:44:19.540 INFO:tasks.rados.rados.0.smithi033.stdout:105: delete oid 30 current snap is 5 2024-09-15T01:44:19.547 INFO:tasks.rados.rados.0.smithi033.stdout:105: done (0 left) 2024-09-15T01:44:19.547 INFO:tasks.rados.rados.0.smithi033.stdout:106: rollback oid 21 current snap is 5 2024-09-15T01:44:19.547 INFO:tasks.rados.rados.0.smithi033.stdout:106: done (0 left) 2024-09-15T01:44:19.547 INFO:tasks.rados.rados.0.smithi033.stdout:107: read oid 11 snap -1 2024-09-15T01:44:19.547 INFO:tasks.rados.rados.0.smithi033.stdout:107: expect deleted 2024-09-15T01:44:19.548 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 24 current snap is 5 2024-09-15T01:44:19.548 INFO:tasks.rados.rados.0.smithi033.stdout:108: seq_num 67 ranges {393216=753664,1146880=376832} 2024-09-15T01:44:19.548 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.548 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.555 INFO:tasks.rados.rados.0.smithi033.stdout:108: writing smithi03327189-24 from 393216 to 1146880 tid 1 2024-09-15T01:44:19.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:19.558 INFO:tasks.rados.rados.0.smithi033.stdout:108: writing smithi03327189-24 from 1146880 to 1523712 tid 2 2024-09-15T01:44:19.559 INFO:tasks.rados.rados.0.smithi033.stdout:107: done (1 left) 2024-09-15T01:44:19.559 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 7 current snap is 5 2024-09-15T01:44:19.560 INFO:tasks.rados.rados.0.smithi033.stdout:109: seq_num 68 ranges {1146880=770048,1916928=376832} 2024-09-15T01:44:19.569 INFO:tasks.rados.rados.0.smithi033.stdout:109: writing smithi03327189-7 from 1146880 to 1916928 tid 1 2024-09-15T01:44:19.571 INFO:tasks.rados.rados.0.smithi033.stdout:109: writing smithi03327189-7 from 1916928 to 2293760 tid 2 2024-09-15T01:44:19.572 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 41 current snap is 5 2024-09-15T01:44:19.572 INFO:tasks.rados.rados.0.smithi033.stdout:108: finishing write tid 1 to smithi03327189-24 2024-09-15T01:44:19.572 INFO:tasks.rados.rados.0.smithi033.stdout:110: seq_num 69 ranges {81920=786432,868352=376832} 2024-09-15T01:44:19.578 INFO:tasks.rados.rados.0.smithi033.stdout:110: writing smithi03327189-41 from 81920 to 868352 tid 1 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:110: writing smithi03327189-41 from 868352 to 1245184 tid 2 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:111: rollback oid 45 current snap is 5 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:108: finishing write tid 2 to smithi03327189-24 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:111: done (3 left) 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:112: snap_create 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:108: finishing write tid 3 to smithi03327189-24 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:108: finishing write tid 4 to smithi03327189-24 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 17 (ObjNum 67 snap 5 seq_num 67) dirty exists 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:108: left oid 24 (ObjNum 67 snap 5 seq_num 67) 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:109: finishing write tid 1 to smithi03327189-7 2024-09-15T01:44:19.581 INFO:tasks.rados.rados.0.smithi033.stdout:109: finishing write tid 2 to smithi03327189-7 2024-09-15T01:44:19.582 INFO:tasks.rados.rados.0.smithi033.stdout:109: finishing write tid 3 to smithi03327189-7 2024-09-15T01:44:19.582 INFO:tasks.rados.rados.0.smithi033.stdout:109: finishing write tid 4 to smithi03327189-7 2024-09-15T01:44:19.582 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 25 (ObjNum 68 snap 5 seq_num 68) dirty exists 2024-09-15T01:44:19.582 INFO:tasks.rados.rados.0.smithi033.stdout:109: left oid 7 (ObjNum 68 snap 5 seq_num 68) 2024-09-15T01:44:19.585 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:19.580+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:19.586 INFO:tasks.rados.rados.0.smithi033.stdout:110: finishing write tid 1 to smithi03327189-41 2024-09-15T01:44:19.588 INFO:tasks.rados.rados.0.smithi033.stdout:110: finishing write tid 2 to smithi03327189-41 2024-09-15T01:44:19.589 INFO:tasks.rados.rados.0.smithi033.stdout:110: finishing write tid 3 to smithi03327189-41 2024-09-15T01:44:19.592 INFO:tasks.rados.rados.0.smithi033.stdout:110: finishing write tid 4 to smithi03327189-41 2024-09-15T01:44:19.592 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 41 v 10 (ObjNum 69 snap 5 seq_num 69) dirty exists 2024-09-15T01:44:19.592 INFO:tasks.rados.rados.0.smithi033.stdout:110: left oid 41 (ObjNum 69 snap 5 seq_num 69) 2024-09-15T01:44:19.686 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:19.680+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:19.700 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.701 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:19.787 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:19.780+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:19.811 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.811 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:19.888 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:19.884+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:19.922 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:19.922 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:19.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:19.989 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:19.987+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:20.033 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.034 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:20.091 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:20.088+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:20.139 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.139 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:20.191 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:20.187+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:20.251 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.251 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.260 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:20.292 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:20.284+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:20.365 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.365 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.376 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:20.393 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:20.388+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:20.470 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.471 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:20.493 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:20.488+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:20.536 INFO:tasks.rados.rados.0.smithi033.stdout:108: done (3 left) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:109: done (2 left) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:110: done (1 left) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:112: done (0 left) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:113: read oid 3 snap -1 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:113: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:114: rollback oid 35 current snap is 6 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 35 to 5 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:115: read oid 24 snap 5 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:115: expect (ObjNum 67 snap 5 seq_num 67) 2024-09-15T01:44:20.537 INFO:tasks.rados.rados.0.smithi033.stdout:116: snap_create 2024-09-15T01:44:20.547 INFO:tasks.rados.rados.0.smithi033.stdout:114: finishing rollback tid 0 to smithi03327189-35 2024-09-15T01:44:20.548 INFO:tasks.rados.rados.0.smithi033.stdout:114: finishing rollback tid 1 to smithi03327189-35 2024-09-15T01:44:20.548 INFO:tasks.rados.rados.0.smithi033.stdout:113: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:20.549 INFO:tasks.rados.rados.0.smithi033.stdout:114: finishing rollback tid 2 to smithi03327189-35 2024-09-15T01:44:20.549 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 19 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:44:20.550 INFO:tasks.rados.rados.0.smithi033.stdout:115: expect (ObjNum 67 snap 5 seq_num 67) 2024-09-15T01:44:20.582 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.582 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.591 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:20.594 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:20.588+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:20.685 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.685 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.694 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:20.695 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:20.687+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:20.792 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.792 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.795 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:20.788+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:20.801 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:20.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:20.896 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:20.887+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:20.898 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:20.899 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:20.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:20.997 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:20.992+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:21.004 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:21.004 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:21.014 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:21.033 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:21.057 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:21.077 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:21.097 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:21.092+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:21.099 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:21.114 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:21.114 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:21.122 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:21.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:21.143 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:21.167 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:21.191 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:21.198 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:21.192+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:21.214 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:21.222 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:21.222 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:21.231 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:21.237 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:21.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:21.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:21.282 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:21.299 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:21.292+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:21.333 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:21.333 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:21.333 INFO:teuthology.orchestra.run.smithi155.stdout: "num_ops": 0 2024-09-15T01:44:21.333 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:21.343 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:21.401 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:21.395+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:21.435 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:21.435 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:21.435 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:21.435 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:21.436 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:21.444 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:21.502 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:21.496+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:21.504 INFO:tasks.rados.rados.0.smithi033.stdout:113: done (3 left) 2024-09-15T01:44:21.505 INFO:tasks.rados.rados.0.smithi033.stdout:114: done (2 left) 2024-09-15T01:44:21.505 INFO:tasks.rados.rados.0.smithi033.stdout:115: done (1 left) 2024-09-15T01:44:21.506 INFO:tasks.rados.rados.0.smithi033.stdout:116: done (0 left) 2024-09-15T01:44:21.506 INFO:tasks.rados.rados.0.smithi033.stdout:117: snap_remove snap 5 2024-09-15T01:44:21.541 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:21.541 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.095365686000001, 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.542 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.543 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:21.544 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:21.546 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.094404159, 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.547 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.548 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:21.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.550 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.091838919000001, 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.551 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:21.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:21.553 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.554 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:21.555 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.090877425999999, 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:21.556 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:21.557 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.558 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.559 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.088794867000001, 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:21.560 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.561 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.562 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.563 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.564 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.08776353, 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.565 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:21.566 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:21.567 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.568 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.086832981000001, 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.569 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:21.570 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.571 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.572 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.573 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.085977573000001, 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.574 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.575 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:21.576 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.577 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 27.083461152000002, 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.578 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:21.579 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.580 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.581 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.582 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 26.531421439999999, 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.583 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.584 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:21.585 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.586 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 26.530007216000001, 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.587 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:21.588 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.589 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:21.591 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 26.529001193999999, 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.594 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:21.595 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 26.527730565999999, 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.596 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:21.597 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.598 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.599 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.600 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 25.972032486, 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.601 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.602 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.603 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:21.604 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=3, to_read={3:720ccbad:::smithi03327189-29:head=0,262144,0}, subchunks={3:720ccbad:::smithi03327189-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 9.0002746439999992, 2024-09-15T01:44:21.605 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033057709999999999, 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.606 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537073+0000", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2750000000000001e-06 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537091+0000", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537118+0000", 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7220999999999999e-05 2024-09-15T01:44:21.607 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537130+0000", 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1551e-05 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537215+0000", 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.5156999999999998e-05 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.608 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.540378+0000", 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0031629000000000002 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=7, to_read={3:7de15fb4:::smithi03327189-49:head=0,262144,0}, subchunks={3:7de15fb4:::smithi03327189-49:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 8.9920412249999995, 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033061729999999999, 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.609 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545307+0000", 2024-09-15T01:44:21.610 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0699999999999999e-06 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545320+0000", 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2867000000000001e-05 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545339+0000", 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9352e-05 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.611 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8869999999999998e-06 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545406+0000", 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7782999999999999e-05 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548612+0000", 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032062140000000002 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.612 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.7s1 33/32 ECSubRead(tid=46, to_read={3:e02ccbe4:::smithi03327189-36:head=0,262144,0}, subchunks={3:e02ccbe4:::smithi03327189-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 8.9861747740000002, 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033818339999999998, 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:21.613 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551173+0000", 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.111e-06 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.614 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551185+0000", 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.205e-05 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551202+0000", 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6971999999999999e-05 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551209+0000", 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5339999999999996e-06 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.615 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551283+0000", 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.4664000000000002e-05 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554554+0000", 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032705030000000001 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=67, to_read={3:ac2d3331:::smithi03327189-18:head=0,262144,0}, subchunks={3:ac2d3331:::smithi03327189-18:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 8.9842230300000008, 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003506547, 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:21.617 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553125+0000", 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2900000000000002e-07 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.618 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553136+0000", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1032999999999999e-05 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553161+0000", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5043000000000001e-05 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553170+0000", 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8729999999999996e-06 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.619 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553235+0000", 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5230999999999999e-05 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556630+0000", 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003395538 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.620 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.cs7 33/32 ECSubRead(tid=68, to_read={3:3584cd82:::smithi03327189-23:head=0,262144,0}, subchunks={3:3584cd82:::smithi03327189-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 8.9841595830000003, 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036502639999999999, 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.621 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553188+0000", 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2599999999999998e-07 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553194+0000", 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3770000000000001e-06 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.622 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553204+0000", 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553210+0000", 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.8850000000000004e-06 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.623 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553352+0000", 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014202600000000001 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556838+0000", 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003485868 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.624 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=71, to_read={3:a22eae17:::smithi03327189-26:head=0,262144,0}, subchunks={3:a22eae17:::smithi03327189-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 8.9718564930000007, 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0030758130000000002, 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:21.625 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565491+0000", 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2699999999999998e-07 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565507+0000", 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6076e-05 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565523+0000", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565529+0000", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565605+0000", 2024-09-15T01:44:21.627 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6344e-05 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.568566+0000", 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00296131 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:21.628 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:21.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:44:21.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:21.631 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:21.628+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:21.731 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:21.723+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:21.739 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:21.739 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:21.739 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:21.739 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:21.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:21.833 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:21.827+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:21.920 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:21.921 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:21.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:21.933 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:21.927+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:22.022 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.022 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:22.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:22.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:22.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.491842713, 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:22.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.489411315, 2024-09-15T01:44:22.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:22.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.035 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.4891725, 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:22.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.489141117000001, 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5099336870000002, 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:22.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5020192849999994, 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:22.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5019900219999993, 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:22.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:22.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4745531760000006, 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:22.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4733864309999998, 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.073 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4730131580000005, 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:22.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4722940599999994, 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4708049540000001, 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00703289, 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547485+0000", 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547486+0000", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3099999999999997e-07 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547490+0000", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1160000000000001e-06 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547502+0000", 2024-09-15T01:44:22.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2284e-05 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547509+0000", 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8709999999999999e-06 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554232+0000", 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067227739999999999 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554518+0000", 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028611400000000002 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4707416710000007, 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006755016, 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547549+0000", 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5400000000000002e-07 2024-09-15T01:44:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547552+0000", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.875e-06 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547560+0000", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2099999999999993e-06 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547565+0000", 2024-09-15T01:44:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.843e-06 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554285+0000", 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067204109999999999 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554304+0000", 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8223000000000002e-05 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4707043580000008, 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074053929999999997, 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547586+0000", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6699999999999999e-07 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547589+0000", 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7970000000000001e-06 2024-09-15T01:44:22.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547597+0000", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7959999999999996e-06 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.547601+0000", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4259999999999996e-06 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554329+0000", 2024-09-15T01:44:22.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067279790000000003 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554991+0000", 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000661928 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=30, last_complete=33'9, committed=1, applied=1))", 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4697815680000001, 2024-09-15T01:44:22.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070855279999999998, 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548509+0000", 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548510+0000", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3689999999999999e-06 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548515+0000", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.259e-06 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548532+0000", 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6586999999999999e-05 2024-09-15T01:44:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548540+0000", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7670000000000003e-06 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555036+0000", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006496155 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.555594+0000", 2024-09-15T01:44:22.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00055839099999999999 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4621908189999999, 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067647840000000002, 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556099+0000", 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556100+0000", 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4499999999999998e-07 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556180+0000", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9863000000000004e-05 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556188+0000", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7590000000000003e-06 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556193+0000", 2024-09-15T01:44:22.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7709999999999997e-06 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562617+0000", 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064236800000000002 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562864+0000", 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000247366 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4616568799999996, 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066845059999999998, 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556633+0000", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556653+0000", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0038999999999999e-05 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556659+0000", 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9510000000000003e-06 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562747+0000", 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060879640000000004 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563318+0000", 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00057055199999999999 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.1s0 33/32 ECSubReadReply(tid=34, attrs_read=0))", 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4614146810000008, 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007571698, 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556876+0000", 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556895+0000", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9117e-05 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.556900+0000", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2440000000000001e-06 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563373+0000", 2024-09-15T01:44:22.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064735360000000002 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564447+0000", 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010738010000000001 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.518082+0000", 2024-09-15T01:44:22.108 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5002080790000001, 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077522930000000004, 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518082+0000", 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518082+0000", 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518083+0000", 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5600000000000005e-07 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518133+0000", 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0683999999999998e-05 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518161+0000", 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7645999999999999e-05 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.518179+0000", 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7520000000000002e-05 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519050+0000", 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087120999999999997 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519076+0000", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6236000000000001e-05 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.525807+0000", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067305660000000003 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.525834+0000", 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7775000000000001e-05 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.4900794899999998, 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:22.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:22.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.117 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:22.119 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:22.112+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:22.220 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:22.211+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:22.264 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:22.265 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:22.321 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:22.316+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:22.372 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:22.373 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:22.373 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:22.373 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:22.422 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:22.420+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9568780649999997, 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9568390600000001, 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9568081159999995, 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9458982329999994, 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0057431050000000001, 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531966+0000", 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.531967+0000", 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000233353 2024-09-15T01:44:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2055999999999999e-05 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.635e-06 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532530+0000", 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031128500000000001 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532541+0000", 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0937999999999999e-05 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537675+0000", 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051336020000000001 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537709+0000", 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4424000000000001e-05 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=62, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9434214549999993, 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055656810000000003, 2024-09-15T01:44:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534443+0000", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534451+0000", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5809999999999999e-06 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534467+0000", 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6405e-05 2024-09-15T01:44:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534475+0000", 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0840000000000003e-06 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534924+0000", 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044986499999999998 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540008+0000", 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050840019999999998 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=63, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9433293060000008, 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058134440000000001, 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.13e-07 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534542+0000", 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3890000000000002e-06 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534557+0000", 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4919000000000001e-05 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534562+0000", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7189999999999998e-06 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540154+0000", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055920869999999999 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540348+0000", 2024-09-15T01:44:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019381699999999999 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=64, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.94329602, 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059416290000000004, 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534573+0000", 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534579+0000", 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6479999999999997e-06 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534584+0000", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6079999999999998e-06 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540397+0000", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058133880000000001 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540510+0000", 2024-09-15T01:44:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001125 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9424169970000005, 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9619238159999997, 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/34 ECSubWrite(tid=56, reqid=osd.3.0:56, at_version=37'14, trim_to=0'0, roll_forward_to=35'10))", 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9500958690000001, 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049829160000000004, 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527768+0000", 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527769+0000", 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9800000000000002e-07 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527795+0000", 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5474e-05 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527809+0000", 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4041e-05 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527816+0000", 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527865+0000", 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9304000000000001e-05 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527876+0000", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.049e-05 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532721+0000", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004845678 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.532751+0000", 2024-09-15T01:44:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9935000000000001e-05 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9555823869999998, 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045657379999999997, 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522282+0000", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522283+0000", 2024-09-15T01:44:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-07 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522313+0000", 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0176999999999999e-05 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522329+0000", 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5685999999999999e-05 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522338+0000", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1230000000000006e-06 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522428+0000", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0483999999999996e-05 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522441+0000", 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3192e-05 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526819+0000", 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043777 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526847+0000", 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8388999999999999e-05 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=59, reqid=osd.3.0:59, at_version=38'16, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9484790390000004, 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062207900000000003, 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529386+0000", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1200000000000002e-07 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529410+0000", 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4343000000000001e-05 2024-09-15T01:44:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529422+0000", 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2128e-05 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529429+0000", 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7750000000000004e-06 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529473+0000", 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3523000000000003e-05 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529479+0000", 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3720000000000001e-06 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535566+0000", 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006087528 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535606+0000", 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9409000000000002e-05 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s7 38/34 ECSubWrite(tid=43, reqid=osd.0.0:42, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9465655240000004, 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004403382, 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531299+0000", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8000000000000004e-07 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531312+0000", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2727e-05 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531340+0000", 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8240000000000001e-05 2024-09-15T01:44:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531348+0000", 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4270000000000002e-06 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531402+0000", 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4184000000000002e-05 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.531412+0000", 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6970000000000005e-06 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535669+0000", 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042573929999999999 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535702+0000", 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2833999999999998e-05 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=60, reqid=osd.3.0:60, at_version=38'18, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9457026270000002, 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060140539999999996, 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8100000000000001e-07 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532195+0000", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2177e-05 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532208+0000", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2991e-05 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532217+0000", 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8309999999999996e-06 2024-09-15T01:44:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532269+0000", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080000000000003e-05 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532279+0000", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6260000000000007e-06 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538153+0000", 2024-09-15T01:44:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058734499999999997 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538176+0000", 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3017999999999999e-05 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s5 38/34 ECSubWrite(tid=61, reqid=osd.3.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=38'8))", 2024-09-15T01:44:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.944477955, 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048073339999999999, 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533386+0000", 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533387+0000", 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.06e-07 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533396+0000", 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7730000000000005e-06 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533405+0000", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9239999999999996e-06 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533409+0000", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6630000000000001e-06 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533473+0000", 2024-09-15T01:44:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3547999999999995e-05 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.533497+0000", 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3768999999999999e-05 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538175+0000", 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046780399999999996 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538194+0000", 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8910999999999999e-05 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9599282950000001, 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9598587629999997, 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9598362270000003, 2024-09-15T01:44:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9495414320000002, 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071464149999999997, 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.06e-07 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528345+0000", 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1920999999999999e-05 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528355+0000", 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4830000000000005e-06 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528360+0000", 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5029999999999997e-06 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528430+0000", 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9640000000000004e-05 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528440+0000", 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0611999999999999e-05 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535446+0000", 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007005173 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535469+0000", 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3577000000000001e-05 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 39/34 ECSubWrite(tid=1, reqid=client.4632.0:258, at_version=0'0, trim_to=0'0, roll_forward_to=39'7))", 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.538587+0000", 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9392770110000002, 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043906809999999996, 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538587+0000", 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538587+0000", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538588+0000", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2900000000000002e-07 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538601+0000", 2024-09-15T01:44:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2880999999999999e-05 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538614+0000", 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2765999999999999e-05 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538620+0000", 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538862+0000", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000241172 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538872+0000", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0638999999999999e-05 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.542934+0000", 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040618779999999997 2024-09-15T01:44:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.542978+0000", 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3650999999999999e-05 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.582 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:22.584 INFO:tasks.rados.rados.0.smithi033.stdout:117: done (0 left) 2024-09-15T01:44:22.584 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 11 current snap is 7 2024-09-15T01:44:22.585 INFO:tasks.rados.rados.0.smithi033.stdout:118: seq_num 70 ranges {0=802816,802816=376832} 2024-09-15T01:44:22.585 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:118: writing smithi03327189-11 from 0 to 802816 tid 1 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:118: writing smithi03327189-11 from 802816 to 1179648 tid 2 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:119: copy_from oid 43 from oid 34 current snap is 7 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:120: read oid 25 snap 6 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:120: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:121: read oid 46 snap -1 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:121: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-15T01:44:22.586 INFO:tasks.rados.rados.0.smithi033.stdout:122: delete oid 28 current snap is 7 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:122: done (4 left) 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:123: rmattr oid 24 current snap is 7 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 41 current snap is 7 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:124: seq_num 72 ranges {1245184=442368,1687552=720896,2408448=49152} 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:124: writing smithi03327189-41 from 1245184 to 1687552 tid 1 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:124: writing smithi03327189-41 from 1687552 to 2408448 tid 2 2024-09-15T01:44:22.588 INFO:tasks.rados.rados.0.smithi033.stdout:124: writing smithi03327189-41 from 2408448 to 2457600 tid 3 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:120: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:118: finishing write tid 1 to smithi03327189-11 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:118: finishing write tid 2 to smithi03327189-11 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:120: done (5 left) 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:125: snap_create 2024-09-15T01:44:22.589 INFO:tasks.rados.rados.0.smithi033.stdout:121: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-15T01:44:22.589 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:22.576+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:22.593 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] out_osds: [11] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:22.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:22.594 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-15T01:44:22.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:118: finishing write tid 3 to smithi03327189-11 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 19 (ObjNum 67 snap 5 seq_num 67) dirty exists 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:119: finishing copy_from racing read to smithi03327189-43 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:118: finishing write tid 4 to smithi03327189-11 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 19 (ObjNum 70 snap 7 seq_num 70) dirty exists 2024-09-15T01:44:22.625 INFO:tasks.rados.rados.0.smithi033.stdout:118: left oid 11 (ObjNum 70 snap 7 seq_num 70) 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:119: finishing copy_from to smithi03327189-43 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 43 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: finishing write tid 1 to smithi03327189-41 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: finishing write tid 2 to smithi03327189-41 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: finishing write tid 3 to smithi03327189-41 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: finishing write tid 4 to smithi03327189-41 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: finishing write tid 5 to smithi03327189-41 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 41 v 15 (ObjNum 72 snap 7 seq_num 72) dirty exists 2024-09-15T01:44:22.626 INFO:tasks.rados.rados.0.smithi033.stdout:124: left oid 41 (ObjNum 72 snap 7 seq_num 72) 2024-09-15T01:44:22.684 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:22.679+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:22.725 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.725 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:22.725 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:22.725 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:22.785 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:22.779+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:22.826 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.826 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:22.826 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:22.826 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:22.826 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:22.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:22.885 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:22.880+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:22.919 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:22.921 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:22.930 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.399650618000001, 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003107659, 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526022+0000", 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526051+0000", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8187e-05 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526061+0000", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0343000000000001e-05 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526068+0000", 2024-09-15T01:44:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7469999999999999e-06 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526304+0000", 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000236197 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526315+0000", 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0946e-05 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529101+0000", 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027858359999999999 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529129+0000", 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8313000000000001e-05 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.390179869000001, 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.4097607930000002, 2024-09-15T01:44:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040598090000000002, 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515912+0000", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0800000000000005e-07 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515965+0000", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3362999999999997e-05 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515984+0000", 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9130000000000001e-05 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516042+0000", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7518000000000001e-05 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516184+0000", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014263599999999999 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516230+0000", 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5240000000000001e-05 2024-09-15T01:44:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519914+0000", 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036847939999999999 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519971+0000", 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6619999999999997e-05 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4064148989999996, 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4062639709999996, 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4036615330000002, 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050197059999999996, 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522012+0000", 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.122e-06 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522045+0000", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3034999999999998e-05 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522067+0000", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2742e-05 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522078+0000", 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0278000000000001e-05 2024-09-15T01:44:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524135+0000", 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020572450000000001 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524153+0000", 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8510000000000001e-05 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527017+0000", 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028639719999999998 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527030+0000", 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2802e-05 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=97, reqid=osd.5.0:95, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.3969473380000004, 2024-09-15T01:44:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041007480000000004, 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528726+0000", 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8700000000000004e-07 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528739+0000", 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3694e-05 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528757+0000", 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501000000000001e-05 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528763+0000", 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7270000000000001e-06 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528985+0000", 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000221955 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528996+0000", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0363000000000001e-05 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532796+0000", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003800257 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532825+0000", 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9363999999999999e-05 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.4039293749999997, 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=34, reqid=client.4632.0:252, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.4014060800000001, 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056439819999999996, 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524309+0000", 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2670000000000003e-05 2024-09-15T01:44:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524323+0000", 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3813e-05 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524331+0000", 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2300000000000008e-06 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524482+0000", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015112700000000001 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524497+0000", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4666e-05 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529866+0000", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053691579999999997 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529910+0000", 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3990999999999999e-05 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=35, reqid=client.4632.0:253, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3994788820000004, 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038225540000000001, 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526194+0000", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9099999999999991e-07 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526216+0000", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2107e-05 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526228+0000", 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.169e-05 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526235+0000", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7309999999999996e-06 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526305+0000", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0108e-05 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526316+0000", 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1347e-05 2024-09-15T01:44:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529991+0000", 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036753570000000002 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530016+0000", 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4223000000000002e-05 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.4s5 43/34 ECSubRead(tid=48, to_read={3:2cc5cf52:::smithi03327189-46:head=0,421888,0}, subchunks={3:2cc5cf52:::smithi03327189-46:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.398765376, 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029064889999999999, 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526914+0000", 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1659999999999997e-06 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526926+0000", 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2221e-05 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526934+0000", 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3459999999999996e-06 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527012+0000", 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8615000000000005e-05 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529813+0000", 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028007430000000001 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.4s5 43/34 ECSubRead(tid=49, to_read={3:2cc5cf52:::smithi03327189-46:head=0,421888,0}, subchunks={3:2cc5cf52:::smithi03327189-46:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.398723614, 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00302615, 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526949+0000", 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9299999999999999e-07 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526955+0000", 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526964+0000", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5720000000000003e-06 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526970+0000", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6380000000000001e-06 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529855+0000", 2024-09-15T01:44:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028850289999999999 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529975+0000", 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011984100000000001 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.7s1 43/34 ECSubRead(tid=73, to_read={3:e5da12c9:::smithi03327189-25:head=0,102400,0}, subchunks={3:e5da12c9:::smithi03327189-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.39662724100000002, 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035108980000000001, 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6800000000000001e-07 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529051+0000", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1720000000000004e-06 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529064+0000", 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2435999999999999e-05 2024-09-15T01:44:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529069+0000", 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2009999999999998e-06 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532343+0000", 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032738820000000001 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532556+0000", 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021283900000000001 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s4 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.39282795300000001, 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046971579999999999, 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532845+0000", 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5600000000000005e-07 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532856+0000", 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1635e-05 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532868+0000", 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1712000000000001e-05 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532874+0000", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3579999999999999e-06 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533006+0000", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013125800000000001 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533016+0000", 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537501+0000", 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044853799999999997 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537541+0000", 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790999999999999e-05 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.39234706400000002, 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042471389999999996, 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533350+0000", 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.427e-05 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533361+0000", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1258e-05 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533367+0000", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.438e-06 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533417+0000", 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9325e-05 2024-09-15T01:44:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533427+0000", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0468999999999999e-05 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537550+0000", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041229780000000002 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537572+0000", 2024-09-15T01:44:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2106000000000001e-05 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=135, reqid=client.4632.0:304, at_version=43'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.37663787700000001, 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059241329999999998, 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549410+0000", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000375464 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549431+0000", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713999999999999e-05 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549438+0000", 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:44:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549499+0000", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1303999999999994e-05 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549514+0000", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4901e-05 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554929+0000", 2024-09-15T01:44:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054145510000000001 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554958+0000", 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9393000000000001e-05 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/34 ECSubWrite(tid=54, reqid=client.4632.0:294, at_version=0'0, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.37419088, 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032223550000000001, 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551482+0000", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551498+0000", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6297e-05 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551514+0000", 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5673999999999999e-05 2024-09-15T01:44:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551522+0000", 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6969999999999993e-06 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551573+0000", 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1700999999999998e-05 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551584+0000", 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554677+0000", 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030922670000000001 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554703+0000", 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6950000000000001e-05 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=136, reqid=client.4632.0:305, at_version=43'13, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.37216125900000002, 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043787469999999997, 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553519+0000", 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2460000000000003e-06 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554227+0000", 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00070824899999999999 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554241+0000", 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4096e-05 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554249+0000", 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554324+0000", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5050999999999997e-05 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554335+0000", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0721e-05 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557866+0000", 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003531621 2024-09-15T01:44:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557890+0000", 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3088999999999999e-05 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=137, reqid=client.4632.0:306, at_version=43'14, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.37053462799999998, 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046968319999999997, 2024-09-15T01:44:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555138+0000", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8599999999999998e-07 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555169+0000", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0957000000000002e-05 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555181+0000", 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1789e-05 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555188+0000", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7479999999999996e-06 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555224+0000", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5938e-05 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555230+0000", 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.998e-06 2024-09-15T01:44:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559806+0000", 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045767990000000003 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559834+0000", 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7917000000000001e-05 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=138, reqid=client.4632.0:307, at_version=43'15, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.36900314000000001, 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031781679999999999, 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556670+0000", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556689+0000", 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8981999999999999e-05 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556706+0000", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7272999999999999e-05 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556713+0000", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8279999999999997e-06 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556755+0000", 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2341e-05 2024-09-15T01:44:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556764+0000", 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8300000000000002e-06 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559837+0000", 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030732849999999998 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559847+0000", 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.022 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:23.023 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-15T01:44:22.501857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26706483,"num_objects":48,"num_object_clones":1,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":230,"num_read_kb":17006,"num_write":336,"num_write_kb":33064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":18043424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":11,"num_per_pool_osds":11,"num_per_pool_omap_osds":11,"kb":1038090240,"kb_used":3330040,"kb_used_data":59076,"kb_used_omap":18,"kb_used_meta":3270765,"kb_avail":1034760200,"statfs":{"total":1063004405760,"available":1059594444800,"internally_reserved":0,"allocated":60493824,"data_stored":59274819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18744,"internal_metadata":3349264072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1032192,"num_objects":-10,"num_object_clones":-6,"num_object_copies":-80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10,"num_whiteouts":0,"num_read":20,"num_read_kb":6614,"num_write":14,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.986903"},"pg_stats":[{"pgid":"3.f","version":"30'7","reported_seq":91,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T01:44:17.518184+0000","last_change":"2024-09-15T01:44:17.518864+0000","last_active":"2024-09-15T01:44:17.518184+0000","last_peered":"2024-09-15T01:44:17.518184+0000","last_clean":"2024-09-15T01:44:17.518184+0000","last_became_active":"2024-09-15T01:44:14.525529+0000","last_became_peered":"2024-09-15T01:44:14.525529+0000","last_unstale":"2024-09-15T01:44:17.518184+0000","last_undegraded":"2024-09-15T01:44:17.518184+0000","last_fullsized":"2024-09-15T01:44:17.518184+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003217,"stat_sum":{"num_bytes":819200,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2165,"num_write":7,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,1,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"}]},{"pgid":"3.e","version":"40'9","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.551625+0000","last_change":"2024-09-15T01:44:19.551625+0000","last_active":"2024-09-15T01:44:19.551625+0000","last_peered":"2024-09-15T01:44:19.551625+0000","last_clean":"2024-09-15T01:44:19.551625+0000","last_became_active":"2024-09-15T01:44:14.525437+0000","last_became_peered":"2024-09-15T01:44:14.525437+0000","last_unstale":"2024-09-15T01:44:19.551625+0000","last_undegraded":"2024-09-15T01:44:19.551625+0000","last_fullsized":"2024-09-15T01:44:19.551625+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.758952+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014782811999999999,"stat_sum":{"num_bytes":475136,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":6,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.d","version":"30'6","reported_seq":84,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T01:44:17.517901+0000","last_change":"2024-09-15T01:44:17.518816+0000","last_active":"2024-09-15T01:44:17.517901+0000","last_peered":"2024-09-15T01:44:17.517901+0000","last_clean":"2024-09-15T01:44:17.517901+0000","last_became_active":"2024-09-15T01:44:14.533366+0000","last_became_peered":"2024-09-15T01:44:14.533366+0000","last_unstale":"2024-09-15T01:44:17.517901+0000","last_undegraded":"2024-09-15T01:44:17.517901+0000","last_fullsized":"2024-09-15T01:44:17.517901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:26.586147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487702,"stat_sum":{"num_bytes":1540096,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,1,3,4,0],"acting":[7,5,9,6,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"}]},{"pgid":"3.c","version":"41'19","reported_seq":112,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T01:44:20.546800+0000","last_change":"2024-09-15T01:44:19.527357+0000","last_active":"2024-09-15T01:44:20.546800+0000","last_peered":"2024-09-15T01:44:20.546800+0000","last_clean":"2024-09-15T01:44:20.546800+0000","last_became_active":"2024-09-15T01:44:14.524938+0000","last_became_peered":"2024-09-15T01:44:14.524938+0000","last_unstale":"2024-09-15T01:44:20.546800+0000","last_undegraded":"2024-09-15T01:44:20.546800+0000","last_fullsized":"2024-09-15T01:44:20.546800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.402812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018836829999999999,"stat_sum":{"num_bytes":983040,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":5,"num_write":14,"num_write_kb":2784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,7],"acting":[5,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.b","version":"40'25","reported_seq":110,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.578727+0000","last_change":"2024-09-15T01:44:19.548331+0000","last_active":"2024-09-15T01:44:19.578727+0000","last_peered":"2024-09-15T01:44:19.578727+0000","last_clean":"2024-09-15T01:44:19.578727+0000","last_became_active":"2024-09-15T01:44:14.534602+0000","last_became_peered":"2024-09-15T01:44:14.534602+0000","last_unstale":"2024-09-15T01:44:19.578727+0000","last_undegraded":"2024-09-15T01:44:19.578727+0000","last_fullsized":"2024-09-15T01:44:19.578727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:17.646194+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01157366,"stat_sum":{"num_bytes":2637824,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":30,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.a","version":"30'6","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T01:44:20.544805+0000","last_change":"2024-09-15T01:44:19.528253+0000","last_active":"2024-09-15T01:44:20.544805+0000","last_peered":"2024-09-15T01:44:20.544805+0000","last_clean":"2024-09-15T01:44:20.544805+0000","last_became_active":"2024-09-15T01:44:07.579889+0000","last_became_peered":"2024-09-15T01:44:07.579889+0000","last_unstale":"2024-09-15T01:44:20.544805+0000","last_undegraded":"2024-09-15T01:44:20.544805+0000","last_fullsized":"2024-09-15T01:44:20.544805+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.248888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001358527,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":6,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.9","version":"38'8","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.538932+0000","last_change":"2024-09-15T01:44:19.538932+0000","last_active":"2024-09-15T01:44:19.538932+0000","last_peered":"2024-09-15T01:44:19.538932+0000","last_clean":"2024-09-15T01:44:19.538932+0000","last_became_active":"2024-09-15T01:44:14.532160+0000","last_became_peered":"2024-09-15T01:44:14.532160+0000","last_unstale":"2024-09-15T01:44:19.538932+0000","last_undegraded":"2024-09-15T01:44:19.538932+0000","last_fullsized":"2024-09-15T01:44:19.538932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.402642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382454,"stat_sum":{"num_bytes":1261568,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,5,1,4,10],"acting":[8,3,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.8","version":"40'10","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.538542+0000","last_change":"2024-09-15T01:44:19.538542+0000","last_active":"2024-09-15T01:44:19.538542+0000","last_peered":"2024-09-15T01:44:19.538542+0000","last_clean":"2024-09-15T01:44:19.538542+0000","last_became_active":"2024-09-15T01:44:14.537683+0000","last_became_peered":"2024-09-15T01:44:14.537683+0000","last_unstale":"2024-09-15T01:44:19.538542+0000","last_undegraded":"2024-09-15T01:44:19.538542+0000","last_fullsized":"2024-09-15T01:44:19.538542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.717904+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012254012999999999,"stat_sum":{"num_bytes":1032192,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":914,"num_write":6,"num_write_kb":1008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,5,10,9],"acting":[8,1,2,4,3,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"1.0","version":"21'92","reported_seq":163,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T01:44:16.520706+0000","last_change":"2024-09-15T01:44:14.520601+0000","last_active":"2024-09-15T01:44:16.520706+0000","last_peered":"2024-09-15T01:44:16.520706+0000","last_clean":"2024-09-15T01:44:16.520706+0000","last_became_active":"2024-09-15T01:44:14.520194+0000","last_became_peered":"2024-09-15T01:44:14.520194+0000","last_unstale":"2024-09-15T01:44:16.520706+0000","last_undegraded":"2024-09-15T01:44:16.520706+0000","last_fullsized":"2024-09-15T01:44:16.520706+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:59.848114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.525434+0000","last_change":"2024-09-15T01:43:57.414021+0000","last_active":"2024-09-15T01:44:19.525434+0000","last_peered":"2024-09-15T01:44:19.525434+0000","last_clean":"2024-09-15T01:44:19.525434+0000","last_became_active":"2024-09-15T01:43:55.397113+0000","last_became_peered":"2024-09-15T01:43:55.397113+0000","last_unstale":"2024-09-15T01:44:19.525434+0000","last_undegraded":"2024-09-15T01:44:19.525434+0000","last_fullsized":"2024-09-15T01:44:19.525434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.064129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056212700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"32'8","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.537807+0000","last_change":"2024-09-15T01:44:19.537807+0000","last_active":"2024-09-15T01:44:19.537807+0000","last_peered":"2024-09-15T01:44:19.537807+0000","last_clean":"2024-09-15T01:44:19.537807+0000","last_became_active":"2024-09-15T01:44:07.579340+0000","last_became_peered":"2024-09-15T01:44:07.579340+0000","last_unstale":"2024-09-15T01:44:19.537807+0000","last_undegraded":"2024-09-15T01:44:19.537807+0000","last_fullsized":"2024-09-15T01:44:19.537807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.070058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010625490000000001,"stat_sum":{"num_bytes":1523712,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1588,"num_write":8,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T01:44:16.520593+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:44:16.520593+0000","last_peered":"2024-09-15T01:44:16.520593+0000","last_clean":"2024-09-15T01:44:16.520593+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:44:16.520593+0000","last_undegraded":"2024-09-15T01:44:16.520593+0000","last_fullsized":"2024-09-15T01:44:16.520593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"40'16","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.539518+0000","last_change":"2024-09-15T01:44:19.539518+0000","last_active":"2024-09-15T01:44:19.539518+0000","last_peered":"2024-09-15T01:44:19.539518+0000","last_clean":"2024-09-15T01:44:19.539518+0000","last_became_active":"2024-09-15T01:44:14.529293+0000","last_became_peered":"2024-09-15T01:44:14.529293+0000","last_unstale":"2024-09-15T01:44:19.539518+0000","last_undegraded":"2024-09-15T01:44:19.539518+0000","last_fullsized":"2024-09-15T01:44:19.539518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:33.100632+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010072034000000001,"stat_sum":{"num_bytes":1261568,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":100,"num_write":10,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,5,9,6,1],"acting":[0,4,10,3,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.536328+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:44:19.536328+0000","last_peered":"2024-09-15T01:44:19.536328+0000","last_clean":"2024-09-15T01:44:19.536328+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:44:19.536328+0000","last_undegraded":"2024-09-15T01:44:19.536328+0000","last_fullsized":"2024-09-15T01:44:19.536328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"30'13","reported_seq":88,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.526920+0000","last_change":"2024-09-15T01:44:19.526920+0000","last_active":"2024-09-15T01:44:19.526920+0000","last_peered":"2024-09-15T01:44:19.526920+0000","last_clean":"2024-09-15T01:44:19.526920+0000","last_became_active":"2024-09-15T01:44:14.525102+0000","last_became_peered":"2024-09-15T01:44:14.525102+0000","last_unstale":"2024-09-15T01:44:19.526920+0000","last_undegraded":"2024-09-15T01:44:19.526920+0000","last_fullsized":"2024-09-15T01:44:19.526920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:06.739129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019455410000000001,"stat_sum":{"num_bytes":1982464,"num_objects":5,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":3704,"num_write":13,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,1,7,8,0,3],"acting":[5,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.2","version":"23'2","reported_seq":35,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.525233+0000","last_change":"2024-09-15T01:43:57.415640+0000","last_active":"2024-09-15T01:44:19.525233+0000","last_peered":"2024-09-15T01:44:19.525233+0000","last_clean":"2024-09-15T01:44:19.525233+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:44:19.525233+0000","last_undegraded":"2024-09-15T01:44:19.525233+0000","last_fullsized":"2024-09-15T01:44:19.525233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.929266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002165143,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'4","reported_seq":91,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.534272+0000","last_change":"2024-09-15T01:44:19.534272+0000","last_active":"2024-09-15T01:44:19.534272+0000","last_peered":"2024-09-15T01:44:19.534272+0000","last_clean":"2024-09-15T01:44:19.534272+0000","last_became_active":"2024-09-15T01:44:14.527181+0000","last_became_peered":"2024-09-15T01:44:14.527181+0000","last_unstale":"2024-09-15T01:44:19.534272+0000","last_undegraded":"2024-09-15T01:44:19.534272+0000","last_fullsized":"2024-09-15T01:44:19.534272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:31.656487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055252100000000005,"stat_sum":{"num_bytes":655360,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":1346,"num_write":4,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,5,9,10],"acting":[4,3,0,6,2,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"3.5","version":"38'34","reported_seq":141,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.527383+0000","last_change":"2024-09-15T01:44:19.527383+0000","last_active":"2024-09-15T01:44:19.527383+0000","last_peered":"2024-09-15T01:44:19.527383+0000","last_clean":"2024-09-15T01:44:19.527383+0000","last_became_active":"2024-09-15T01:44:14.526182+0000","last_became_peered":"2024-09-15T01:44:14.526182+0000","last_unstale":"2024-09-15T01:44:19.527383+0000","last_undegraded":"2024-09-15T01:44:19.527383+0000","last_fullsized":"2024-09-15T01:44:19.527383+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.259816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016792420000000001,"stat_sum":{"num_bytes":5079040,"num_objects":8,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":20,"num_read_kb":2313,"num_write":24,"num_write_kb":4960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5160960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,8,10],"acting":[5,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.528475+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:44:19.528475+0000","last_peered":"2024-09-15T01:44:19.528475+0000","last_clean":"2024-09-15T01:44:19.528475+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:44:19.528475+0000","last_undegraded":"2024-09-15T01:44:19.528475+0000","last_fullsized":"2024-09-15T01:44:19.528475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"40'17","reported_seq":113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T01:44:20.542071+0000","last_change":"2024-09-15T01:44:19.530076+0000","last_active":"2024-09-15T01:44:20.542071+0000","last_peered":"2024-09-15T01:44:20.542071+0000","last_clean":"2024-09-15T01:44:20.542071+0000","last_became_active":"2024-09-15T01:44:14.526114+0000","last_became_peered":"2024-09-15T01:44:14.526114+0000","last_unstale":"2024-09-15T01:44:20.542071+0000","last_undegraded":"2024-09-15T01:44:20.542071+0000","last_fullsized":"2024-09-15T01:44:20.542071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.424117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032828940000000002,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4502,"num_write":35,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,5,10,2,6,0],"acting":[1,8,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.529434+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:44:19.529434+0000","last_peered":"2024-09-15T01:44:19.529434+0000","last_clean":"2024-09-15T01:44:19.529434+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:44:19.529434+0000","last_undegraded":"2024-09-15T01:44:19.529434+0000","last_fullsized":"2024-09-15T01:44:19.529434+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'8","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.538237+0000","last_change":"2024-09-15T01:44:19.538237+0000","last_active":"2024-09-15T01:44:19.538237+0000","last_peered":"2024-09-15T01:44:19.538237+0000","last_clean":"2024-09-15T01:44:19.538237+0000","last_became_active":"2024-09-15T01:44:14.534165+0000","last_became_peered":"2024-09-15T01:44:14.534165+0000","last_unstale":"2024-09-15T01:44:19.538237+0000","last_undegraded":"2024-09-15T01:44:19.538237+0000","last_fullsized":"2024-09-15T01:44:19.538237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.075163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081407599999999995,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":27,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.530261+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:44:19.530261+0000","last_peered":"2024-09-15T01:44:19.530261+0000","last_clean":"2024-09-15T01:44:19.530261+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:44:19.530261+0000","last_undegraded":"2024-09-15T01:44:19.530261+0000","last_fullsized":"2024-09-15T01:44:19.530261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"40'10","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.537657+0000","last_change":"2024-09-15T01:44:19.533881+0000","last_active":"2024-09-15T01:44:19.537657+0000","last_peered":"2024-09-15T01:44:19.537657+0000","last_clean":"2024-09-15T01:44:19.537657+0000","last_became_active":"2024-09-15T01:44:07.578405+0000","last_became_peered":"2024-09-15T01:44:07.578405+0000","last_unstale":"2024-09-15T01:44:19.537657+0000","last_undegraded":"2024-09-15T01:44:19.537657+0000","last_fullsized":"2024-09-15T01:44:19.537657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:31.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040034010000000002,"stat_sum":{"num_bytes":376832,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":10,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T01:44:19.527154+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:44:19.527154+0000","last_peered":"2024-09-15T01:44:19.527154+0000","last_clean":"2024-09-15T01:44:19.527154+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:44:19.527154+0000","last_undegraded":"2024-09-15T01:44:19.527154+0000","last_fullsized":"2024-09-15T01:44:19.527154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":26116096,"num_objects":44,"num_object_clones":1,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":154,"num_read_kb":16942,"num_write":211,"num_write_kb":30912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":17514496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":59195392,"data_stored":60370944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411337,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302788,"kb_used_data":5424,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069052,"statfs":{"total":96636764160,"available":96326709248,"internally_reserved":0,"allocated":5554176,"data_stored":5459295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444040,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444041,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302964,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068876,"statfs":{"total":96636764160,"available":96326529024,"internally_reserved":0,"allocated":5738496,"data_stored":5672405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]}]},{"osd":9,"up_from":17,"seq":73014444040,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":5868,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":6008832,"data_stored":5942741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444040,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302920,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068920,"statfs":{"total":96636764160,"available":96326574080,"internally_reserved":0,"allocated":5693440,"data_stored":5475797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]}]},{"osd":6,"up_from":17,"seq":73014444041,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":5912,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":6053888,"data_stored":5984245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]}]},{"osd":5,"up_from":17,"seq":73014444041,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":5104,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":5226496,"data_stored":5156328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444040,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301684,"kb_used_data":4324,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070156,"statfs":{"total":96636764160,"available":96327839744,"internally_reserved":0,"allocated":4427776,"data_stored":4210133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":3,"up_from":17,"seq":73014444040,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303072,"kb_used_data":5712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068768,"statfs":{"total":96636764160,"available":96326418432,"internally_reserved":0,"allocated":5849088,"data_stored":5782997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0830000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1930000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2570000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1560000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444041,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301584,"kb_used_data":4224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070256,"statfs":{"total":96636764160,"available":96327942144,"internally_reserved":0,"allocated":4325376,"data_stored":4259285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]}]},{"osd":1,"up_from":17,"seq":73014444040,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304072,"kb_used_data":6712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067768,"statfs":{"total":96636764160,"available":96325394432,"internally_reserved":0,"allocated":6873088,"data_stored":6651368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":0,"up_from":17,"seq":73014444040,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301992,"kb_used_data":4632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069848,"statfs":{"total":96636764160,"available":96327524352,"internally_reserved":0,"allocated":4743168,"data_stored":4680225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4288512,"data_stored":4440064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6410240,"data_stored":6410240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3866624,"data_stored":4018176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5390336,"data_stored":5541888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3969024,"data_stored":3969024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4763648,"data_stored":4915200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5062656,"data_stored":5214208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4599808,"data_stored":4714496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5550080,"data_stored":5701632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":5279744,"data_stored":5431296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:23.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-15T01:44:23.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:23.028 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:23.016+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:23.124 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:23.120+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:23.175 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:23.175 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:23.175 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:23.176 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:23.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:23.225 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:23.223+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:23.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:23.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:23.326 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:23.323+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:23.413 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.900295588000001, 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.414 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:23.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.900241349, 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.900222210999999, 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:206 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~606208] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.889903399, 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062804979999999998, 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 206 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.428 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519793+0000", 2024-09-15T01:44:23.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0190000000000001e-06 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520167+0000", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037447000000000001 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520180+0000", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3101e-05 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520186+0000", 2024-09-15T01:44:23.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9989999999999997e-06 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521198+0000", 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001011807 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521308+0000", 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110059 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522439+0000", 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011307229999999999 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525693+0000", 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003254159 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526006+0000", 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000312971 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526071+0000", 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5190000000000004e-05 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:207 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.888200467000001, 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055629219999999997, 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 207 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521495+0000", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.618e-06 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521721+0000", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022585199999999999 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521741+0000", 2024-09-15T01:44:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0496000000000002e-05 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521749+0000", 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2620000000000001e-06 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523331+0000", 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015822919999999999 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523435+0000", 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001044 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524251+0000", 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000815367 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526200+0000", 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019496120000000001 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526995+0000", 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079458399999999998 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527056+0000", 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1439000000000006e-05 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:208 3.2s0 3:47035a48:::smithi03327189-20:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.887934626, 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056281209999999998, 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 208 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521760+0000", 2024-09-15T01:44:23.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521765+0000", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521774+0000", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8240000000000006e-06 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521778+0000", 2024-09-15T01:44:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3760000000000001e-06 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524887+0000", 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031084820000000001 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525036+0000", 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000149736 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525431+0000", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039431399999999999 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527107+0000", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001676147 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527310+0000", 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000203632 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527387+0000", 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6938000000000001e-05 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:209 3.2s0 3:47035a48:::smithi03327189-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.887911748000001, 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079111819999999992, 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 209 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521783+0000", 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521784+0000", 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.736e-06 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521791+0000", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0849999999999999e-06 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521803+0000", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526224+0000", 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004421431 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526350+0000", 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012618500000000001 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527766+0000", 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014151299999999999 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527853+0000", 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7435999999999994e-05 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529693+0000", 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001840436 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.884419742, 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005452106, 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525306+0000", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1396000000000002e-05 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525315+0000", 2024-09-15T01:44:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525321+0000", 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.229e-06 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525509+0000", 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018865900000000001 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525520+0000", 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0285000000000001e-05 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530704+0000", 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051837840000000003 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530726+0000", 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2795000000000001e-05 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0, backfill_or_async_recovery))", 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.877519198, 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083125319999999992, 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532176+0000", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:44:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2191000000000001e-05 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6229999999999999e-06 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532292+0000", 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3135999999999999e-05 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532304+0000", 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2481e-05 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540422+0000", 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081175480000000005 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540487+0000", 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5278999999999996e-05 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.874420089999999, 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077245059999999999, 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535275+0000", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000001e-07 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535296+0000", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1590000000000002e-05 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535309+0000", 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2247e-05 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535315+0000", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7870000000000004e-06 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535399+0000", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3398000000000003e-05 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535410+0000", 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542978+0000", 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075673989999999998 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542998+0000", 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0469999999999999e-05 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4, backfill_or_async_recovery))", 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.872855653, 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061654960000000003, 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536839+0000", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536859+0000", 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0585999999999999e-05 2024-09-15T01:44:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536869+0000", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3989999999999993e-06 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536878+0000", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6180000000000001e-06 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536929+0000", 2024-09-15T01:44:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0285999999999999e-05 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536939+0000", 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0472000000000001e-05 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542974+0000", 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006034569 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543004+0000", 2024-09-15T01:44:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0196999999999999e-05 2024-09-15T01:44:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8890429129999999, 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065983789999999997, 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520652+0000", 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0330000000000001e-06 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520685+0000", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3210999999999998e-05 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520702+0000", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6880000000000001e-05 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520712+0000", 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0003000000000001e-05 2024-09-15T01:44:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520819+0000", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106947 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520835+0000", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5647000000000001e-05 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527222+0000", 2024-09-15T01:44:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063877339999999999 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527249+0000", 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6923999999999998e-05 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 38/34 ECSubWrite(tid=42, reqid=osd.0.0:42, at_version=38'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8850805250000002, 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060356519999999999, 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524614+0000", 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7999999999999996e-07 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524667+0000", 2024-09-15T01:44:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2803000000000002e-05 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524683+0000", 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6623e-05 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524692+0000", 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5169999999999999e-06 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524799+0000", 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010671300000000001 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524810+0000", 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1661999999999999e-05 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530622+0000", 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058122499999999997 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530649+0000", 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6604000000000001e-05 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es6 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8825482039999999, 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052142990000000004, 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5699999999999998e-07 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527172+0000", 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6142000000000001e-05 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527191+0000", 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8192000000000001e-05 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527198+0000", 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527308+0000", 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109604 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527319+0000", 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1124999999999999e-05 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.532330+0000", 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050108569999999996 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.532360+0000", 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0133000000000001e-05 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.89124731099999999, 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.89117964900000002, 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.89115390800000005, 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.87808309900000003, 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.87704119400000002, 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.87622038899999999, 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:23.510 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:23.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:23.514 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:23.514 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:23.514 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:23.514 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:23.515 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:23.508+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:23.518 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:23.541 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:23.554 INFO:tasks.rados.rados.0.smithi033.stdout:118: done (5 left) 2024-09-15T01:44:23.554 INFO:tasks.rados.rados.0.smithi033.stdout:119: done (4 left) 2024-09-15T01:44:23.554 INFO:tasks.rados.rados.0.smithi033.stdout:121: done (3 left) 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:123: done (2 left) 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:124: done (1 left) 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:125: done (0 left) 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:126: rollback oid 36 current snap is 8 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 36 to 7 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 39 current snap is 8 2024-09-15T01:44:23.555 INFO:tasks.rados.rados.0.smithi033.stdout:127: seq_num 73 ranges {638976=458752,1097728=475136,1572864=294912} 2024-09-15T01:44:23.567 INFO:tasks.rados.rados.0.smithi033.stdout:127: writing smithi03327189-39 from 638976 to 1097728 tid 1 2024-09-15T01:44:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 3 pgp_num to 16 2024-09-15T01:44:23.570 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:23.570 INFO:tasks.rados.rados.0.smithi033.stdout:127: writing smithi03327189-39 from 1097728 to 1572864 tid 2 2024-09-15T01:44:23.572 INFO:tasks.rados.rados.0.smithi033.stdout:127: writing smithi03327189-39 from 1572864 to 1867776 tid 3 2024-09-15T01:44:23.572 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 23 current snap is 8 2024-09-15T01:44:23.572 INFO:tasks.rados.rados.0.smithi033.stdout:126: finishing rollback tid 0 to smithi03327189-36 2024-09-15T01:44:23.572 INFO:tasks.rados.rados.0.smithi033.stdout:128: seq_num 74 ranges {376832=475136,851968=622592,1474560=147456} 2024-09-15T01:44:23.577 INFO:tasks.rados.rados.0.smithi033.stdout:128: writing smithi03327189-23 from 376832 to 851968 tid 1 2024-09-15T01:44:23.582 INFO:tasks.rados.rados.0.smithi033.stdout:128: writing smithi03327189-23 from 851968 to 1474560 tid 2 2024-09-15T01:44:23.583 INFO:tasks.rados.rados.0.smithi033.stdout:128: writing smithi03327189-23 from 1474560 to 1622016 tid 3 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 14 current snap is 8 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:126: finishing rollback tid 1 to smithi03327189-36 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:126: finishing rollback tid 2 to smithi03327189-36 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 36 v 12 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:127: finishing write tid 1 to smithi03327189-39 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:127: finishing write tid 2 to smithi03327189-39 2024-09-15T01:44:23.584 INFO:tasks.rados.rados.0.smithi033.stdout:129: seq_num 75 ranges {1228800=491520,1720320=770048} 2024-09-15T01:44:23.594 INFO:tasks.rados.rados.0.smithi033.stdout:129: writing smithi03327189-14 from 1228800 to 1720320 tid 1 2024-09-15T01:44:23.597 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:23.599 INFO:tasks.rados.rados.0.smithi033.stdout:129: writing smithi03327189-14 from 1720320 to 2490368 tid 2 2024-09-15T01:44:23.599 INFO:tasks.rados.rados.0.smithi033.stdout:126: done (3 left) 2024-09-15T01:44:23.599 INFO:tasks.rados.rados.0.smithi033.stdout:130: rmattr oid 11 current snap is 8 2024-09-15T01:44:23.599 INFO:tasks.rados.rados.0.smithi033.stdout:130: done (3 left) 2024-09-15T01:44:23.599 INFO:tasks.rados.rados.0.smithi033.stdout:131: snap_remove snap 6 2024-09-15T01:44:23.613 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:23.608+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:23.621 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:23.637 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:23.638 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:23.642 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:23.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:23.665 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:23.715 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:23.708+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:23.745 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:23.745 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:23.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:23.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:23.808+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:23.857 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:23.858 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:23.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:23.916 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:23.908+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:23.967 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:23.967 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:23.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:24.016 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:24.012+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:24.118 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:24.111+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:24.126 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.126 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.135 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:24.219 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:24.215+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.240 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:24.320 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:24.312+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:24.352 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.352 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:24.421 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:24.416+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:24.467 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.467 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:24.525 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:24.520+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:131: done (3 left) 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:132: rollback oid 20 current snap is 8 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 20 to 7 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:127: finishing write tid 3 to smithi03327189-39 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:127: finishing write tid 4 to smithi03327189-39 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:127: finishing write tid 5 to smithi03327189-39 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 39 v 18 (ObjNum 73 snap 8 seq_num 73) dirty exists 2024-09-15T01:44:24.567 INFO:tasks.rados.rados.0.smithi033.stdout:127: left oid 39 (ObjNum 73 snap 8 seq_num 73) 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: finishing write tid 1 to smithi03327189-23 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: finishing write tid 2 to smithi03327189-23 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: finishing write tid 3 to smithi03327189-23 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: finishing write tid 4 to smithi03327189-23 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: finishing write tid 5 to smithi03327189-23 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 23 v 26 (ObjNum 74 snap 8 seq_num 74) dirty exists 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: left oid 23 (ObjNum 74 snap 8 seq_num 74) 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:129: finishing write tid 1 to smithi03327189-14 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:129: finishing write tid 2 to smithi03327189-14 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:127: done (3 left) 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:128: done (2 left) 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 26 current snap is 8 2024-09-15T01:44:24.568 INFO:tasks.rados.rados.0.smithi033.stdout:133: seq_num 76 ranges {425984=507904,933888=507904,1441792=262144} 2024-09-15T01:44:24.580 INFO:tasks.rados.rados.0.smithi033.stdout:133: writing smithi03327189-26 from 425984 to 933888 tid 1 2024-09-15T01:44:24.585 INFO:tasks.rados.rados.0.smithi033.stdout:133: writing smithi03327189-26 from 933888 to 1441792 tid 2 2024-09-15T01:44:24.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:133: writing smithi03327189-26 from 1441792 to 1703936 tid 3 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:134: copy_from oid 25 from oid 22 current snap is 8 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:129: finishing write tid 3 to smithi03327189-14 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:129: finishing write tid 4 to smithi03327189-14 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 23 (ObjNum 75 snap 8 seq_num 75) dirty exists 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:129: left oid 14 (ObjNum 75 snap 8 seq_num 75) 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:129: done (3 left) 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:135: read oid 43 snap -1 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:135: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:136: rollback oid 8 current snap is 8 2024-09-15T01:44:24.589 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 8 to 7 2024-09-15T01:44:24.590 INFO:tasks.rados.rados.0.smithi033.stdout:132: finishing rollback tid 0 to smithi03327189-20 2024-09-15T01:44:24.590 INFO:tasks.rados.rados.0.smithi033.stdout:132: finishing rollback tid 1 to smithi03327189-20 2024-09-15T01:44:24.590 INFO:tasks.rados.rados.0.smithi033.stdout:137: delete oid 44 current snap is 8 2024-09-15T01:44:24.590 INFO:tasks.rados.rados.0.smithi033.stdout:132: finishing rollback tid 2 to smithi03327189-20 2024-09-15T01:44:24.591 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 12 (ObjNum 58 snap 2 seq_num 58) dirty exists 2024-09-15T01:44:24.591 INFO:tasks.rados.rados.0.smithi033.stdout:134: finishing copy_from to smithi03327189-25 2024-09-15T01:44:24.591 INFO:tasks.rados.rados.0.smithi033.stdout:134: got expected ENOENT (src dne) 2024-09-15T01:44:24.592 INFO:tasks.rados.rados.0.smithi033.stdout:134: finishing copy_from racing read to smithi03327189-25 2024-09-15T01:44:24.594 INFO:tasks.rados.rados.0.smithi033.stdout:133: finishing write tid 1 to smithi03327189-26 2024-09-15T01:44:24.596 INFO:tasks.rados.rados.0.smithi033.stdout:132: done (5 left) 2024-09-15T01:44:24.596 INFO:tasks.rados.rados.0.smithi033.stdout:134: done (4 left) 2024-09-15T01:44:24.596 INFO:tasks.rados.rados.0.smithi033.stdout:137: done (3 left) 2024-09-15T01:44:24.597 INFO:tasks.rados.rados.0.smithi033.stdout:138: delete oid 47 current snap is 8 2024-09-15T01:44:24.599 INFO:tasks.rados.rados.0.smithi033.stdout:133: finishing write tid 2 to smithi03327189-26 2024-09-15T01:44:24.599 INFO:tasks.rados.rados.0.smithi033.stdout:136: finishing rollback tid 1 to smithi03327189-8 2024-09-15T01:44:24.599 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 8 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T01:44:24.599 INFO:tasks.rados.rados.0.smithi033.stdout:133: finishing write tid 3 to smithi03327189-26 2024-09-15T01:44:24.599 INFO:tasks.rados.rados.0.smithi033.stdout:133: finishing write tid 4 to smithi03327189-26 2024-09-15T01:44:24.600 INFO:tasks.rados.rados.0.smithi033.stdout:133: finishing write tid 5 to smithi03327189-26 2024-09-15T01:44:24.600 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 26 v 39 (ObjNum 76 snap 8 seq_num 76) dirty exists 2024-09-15T01:44:24.600 INFO:tasks.rados.rados.0.smithi033.stdout:133: left oid 26 (ObjNum 76 snap 8 seq_num 76) 2024-09-15T01:44:24.602 INFO:tasks.rados.rados.0.smithi033.stdout:135: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:133: done (3 left) 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:135: done (2 left) 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:136: done (1 left) 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:138: done (0 left) 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 9 current snap is 8 2024-09-15T01:44:24.617 INFO:tasks.rados.rados.0.smithi033.stdout:139: seq_num 79 ranges {147456=573440,720896=425984,1146880=344064} 2024-09-15T01:44:24.621 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:24.615+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:24.622 INFO:tasks.rados.rados.0.smithi033.stdout:139: writing smithi03327189-9 from 147456 to 720896 tid 1 2024-09-15T01:44:24.626 INFO:tasks.rados.rados.0.smithi033.stdout:139: writing smithi03327189-9 from 720896 to 1146880 tid 2 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:139: writing smithi03327189-9 from 1146880 to 1490944 tid 3 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:140: rollback oid 26 current snap is 8 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 26 to 7 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:139: finishing write tid 1 to smithi03327189-9 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:141: copy_from oid 35 from oid 14 current snap is 8 2024-09-15T01:44:24.632 INFO:tasks.rados.rados.0.smithi033.stdout:142: delete oid 19 current snap is 8 2024-09-15T01:44:24.638 INFO:tasks.rados.rados.0.smithi033.stdout:139: finishing write tid 2 to smithi03327189-9 2024-09-15T01:44:24.638 INFO:tasks.rados.rados.0.smithi033.stdout:140: finishing rollback tid 0 to smithi03327189-26 2024-09-15T01:44:24.638 INFO:tasks.rados.rados.0.smithi033.stdout:142: done (3 left) 2024-09-15T01:44:24.638 INFO:tasks.rados.rados.0.smithi033.stdout:143: rollback oid 46 current snap is 8 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 46 to 7 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:144: read oid 11 snap -1 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:144: expect (ObjNum 70 snap 7 seq_num 70) 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:145: rollback oid 33 current snap is 8 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 33 to 7 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 32 current snap is 8 2024-09-15T01:44:24.639 INFO:tasks.rados.rados.0.smithi033.stdout:146: seq_num 81 ranges {524288=606208,1130496=720896,1851392=49152} 2024-09-15T01:44:24.645 INFO:tasks.rados.rados.0.smithi033.stdout:146: writing smithi03327189-32 from 524288 to 1130496 tid 1 2024-09-15T01:44:24.650 INFO:tasks.rados.rados.0.smithi033.stdout:146: writing smithi03327189-32 from 1130496 to 1851392 tid 2 2024-09-15T01:44:24.650 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.650 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.651 INFO:tasks.rados.rados.0.smithi033.stdout:146: writing smithi03327189-32 from 1851392 to 1900544 tid 3 2024-09-15T01:44:24.651 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 41 current snap is 8 2024-09-15T01:44:24.651 INFO:tasks.rados.rados.0.smithi033.stdout:140: finishing rollback tid 1 to smithi03327189-26 2024-09-15T01:44:24.651 INFO:tasks.rados.rados.0.smithi033.stdout:147: seq_num 82 ranges {2457600=622592,3080192=475136,3555328=294912} 2024-09-15T01:44:24.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:24.670 INFO:tasks.rados.rados.0.smithi033.stdout:147: writing smithi03327189-41 from 2457600 to 3080192 tid 1 2024-09-15T01:44:24.673 INFO:tasks.rados.rados.0.smithi033.stdout:147: writing smithi03327189-41 from 3080192 to 3555328 tid 2 2024-09-15T01:44:24.675 INFO:tasks.rados.rados.0.smithi033.stdout:147: writing smithi03327189-41 from 3555328 to 3850240 tid 3 2024-09-15T01:44:24.675 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 10 current snap is 8 2024-09-15T01:44:24.675 INFO:tasks.rados.rados.0.smithi033.stdout:148: seq_num 83 ranges {163840=638976,802816=622592,1425408=147456} 2024-09-15T01:44:24.680 INFO:tasks.rados.rados.0.smithi033.stdout:148: writing smithi03327189-10 from 163840 to 802816 tid 1 2024-09-15T01:44:24.684 INFO:tasks.rados.rados.0.smithi033.stdout:148: writing smithi03327189-10 from 802816 to 1425408 tid 2 2024-09-15T01:44:24.686 INFO:tasks.rados.rados.0.smithi033.stdout:148: writing smithi03327189-10 from 1425408 to 1572864 tid 3 2024-09-15T01:44:24.686 INFO:tasks.rados.rados.0.smithi033.stdout:149: copy_from oid 20 from oid 28 current snap is 8 2024-09-15T01:44:24.686 INFO:tasks.rados.rados.0.smithi033.stdout:144: expect (ObjNum 70 snap 7 seq_num 70) 2024-09-15T01:44:24.722 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:24.715+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:139: finishing write tid 3 to smithi03327189-9 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:139: finishing write tid 4 to smithi03327189-9 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:140: finishing rollback tid 2 to smithi03327189-26 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 26 v 44 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:143: finishing rollback tid 0 to smithi03327189-46 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:139: finishing write tid 5 to smithi03327189-9 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 9 v 26 (ObjNum 79 snap 8 seq_num 79) dirty exists 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:139: left oid 9 (ObjNum 79 snap 8 seq_num 79) 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:145: finishing rollback tid 0 to smithi03327189-33 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:143: finishing rollback tid 1 to smithi03327189-46 2024-09-15T01:44:24.724 INFO:tasks.rados.rados.0.smithi033.stdout:145: finishing rollback tid 1 to smithi03327189-33 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:143: finishing rollback tid 2 to smithi03327189-46 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 46 v 30 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:141: finishing copy_from racing read to smithi03327189-35 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:145: finishing rollback tid 2 to smithi03327189-33 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 33 v 16 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:146: finishing write tid 1 to smithi03327189-32 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:146: finishing write tid 2 to smithi03327189-32 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:146: finishing write tid 3 to smithi03327189-32 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:139: done (9 left) 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:140: done (8 left) 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:143: done (7 left) 2024-09-15T01:44:24.725 INFO:tasks.rados.rados.0.smithi033.stdout:144: done (6 left) 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:145: done (5 left) 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 12 current snap is 8 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:146: finishing write tid 4 to smithi03327189-32 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:141: finishing copy_from to smithi03327189-35 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 23 (ObjNum 75 snap 8 seq_num 75) dirty exists 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:146: finishing write tid 5 to smithi03327189-32 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 32 v 49 (ObjNum 81 snap 8 seq_num 81) dirty exists 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:146: left oid 32 (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: finishing write tid 1 to smithi03327189-41 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: finishing write tid 2 to smithi03327189-41 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: finishing write tid 3 to smithi03327189-41 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: finishing write tid 4 to smithi03327189-41 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: finishing write tid 5 to smithi03327189-41 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 41 v 22 (ObjNum 82 snap 8 seq_num 82) dirty exists 2024-09-15T01:44:24.726 INFO:tasks.rados.rados.0.smithi033.stdout:147: left oid 41 (ObjNum 82 snap 8 seq_num 82) 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:149: finishing copy_from to smithi03327189-20 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:149: got expected ENOENT (src dne) 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:149: finishing copy_from racing read to smithi03327189-20 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: finishing write tid 1 to smithi03327189-10 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: finishing write tid 2 to smithi03327189-10 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: finishing write tid 3 to smithi03327189-10 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: finishing write tid 4 to smithi03327189-10 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: finishing write tid 5 to smithi03327189-10 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 10 v 15 (ObjNum 83 snap 8 seq_num 83) dirty exists 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:148: left oid 10 (ObjNum 83 snap 8 seq_num 83) 2024-09-15T01:44:24.727 INFO:tasks.rados.rados.0.smithi033.stdout:150: seq_num 84 ranges {196608=655360,851968=638976,1490944=131072} 2024-09-15T01:44:24.730 INFO:tasks.rados.rados.0.smithi033.stdout:150: writing smithi03327189-12 from 196608 to 851968 tid 1 2024-09-15T01:44:24.736 INFO:tasks.rados.rados.0.smithi033.stdout:150: writing smithi03327189-12 from 851968 to 1490944 tid 2 2024-09-15T01:44:24.737 INFO:tasks.rados.rados.0.smithi033.stdout:150: writing smithi03327189-12 from 1490944 to 1622016 tid 3 2024-09-15T01:44:24.737 INFO:tasks.rados.rados.0.smithi033.stdout:141: done (5 left) 2024-09-15T01:44:24.737 INFO:tasks.rados.rados.0.smithi033.stdout:146: done (4 left) 2024-09-15T01:44:24.737 INFO:tasks.rados.rados.0.smithi033.stdout:147: done (3 left) 2024-09-15T01:44:24.740 INFO:tasks.rados.rados.0.smithi033.stdout:148: done (2 left) 2024-09-15T01:44:24.740 INFO:tasks.rados.rados.0.smithi033.stdout:149: done (1 left) 2024-09-15T01:44:24.740 INFO:tasks.rados.rados.0.smithi033.stdout:151: rmattr oid 39 current snap is 8 2024-09-15T01:44:24.741 INFO:tasks.rados.rados.0.smithi033.stdout:151: done (1 left) 2024-09-15T01:44:24.741 INFO:tasks.rados.rados.0.smithi033.stdout:152: rmattr oid 30 current snap is 8 2024-09-15T01:44:24.741 INFO:tasks.rados.rados.0.smithi033.stdout:152: done (1 left) 2024-09-15T01:44:24.741 INFO:tasks.rados.rados.0.smithi033.stdout:153: snap_remove snap 7 2024-09-15T01:44:24.766 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.766 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:24.822 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:24.816+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:24.872 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.873 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.881 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:24.924 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:24.916+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:24.985 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:24.985 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:24.994 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:25.024 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:25.020+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:25.106 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.106 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:25.126 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:25.120+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:25.215 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.215 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:25.226 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:25.219+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:25.324 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.324 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.327 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:25.320+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:25.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:25.425 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.425 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.429 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:25.424+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:25.434 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:25.529 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:25.524+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:25.533 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.533 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.542 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:153: done (1 left) 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:154: read oid 47 snap -1 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:154: expect deleted 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:155: rollback oid 1 current snap is 8 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:155: done (2 left) 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:156: snap_create 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:150: finishing write tid 1 to smithi03327189-12 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:150: finishing write tid 2 to smithi03327189-12 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:150: finishing write tid 3 to smithi03327189-12 2024-09-15T01:44:25.576 INFO:tasks.rados.rados.0.smithi033.stdout:150: finishing write tid 4 to smithi03327189-12 2024-09-15T01:44:25.577 INFO:tasks.rados.rados.0.smithi033.stdout:150: finishing write tid 5 to smithi03327189-12 2024-09-15T01:44:25.577 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 13 (ObjNum 84 snap 8 seq_num 84) dirty exists 2024-09-15T01:44:25.577 INFO:tasks.rados.rados.0.smithi033.stdout:150: left oid 12 (ObjNum 84 snap 8 seq_num 84) 2024-09-15T01:44:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:44:25.630 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:25.624+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:25.654 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.654 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.663 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:25.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:25.730 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:25.724+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:25.757 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.757 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:25.831 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:25.824+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:25.854 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:25.867 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.867 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.873 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:25.876 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:25.893 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:25.914 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:25.932 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:25.924+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:25.936 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:25.976 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:25.977 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:25.983 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:25.985 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:26.006 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:26.033 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:26.027+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:26.053 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:26.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:26.084 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:26.085 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:26.094 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:26.137 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:26.131+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:26.235 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:26.228+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:26.241 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:26.241 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [], 2024-09-15T01:44:26.241 INFO:teuthology.orchestra.run.smithi155.stdout: "complaint_time": 30, 2024-09-15T01:44:26.241 INFO:teuthology.orchestra.run.smithi155.stdout: "num_blocked_ops": 0 2024-09-15T01:44:26.241 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:26.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:26.336 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:26.332+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:26.351 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.904592314999999, 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.352 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.353 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.354 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:26.355 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:26.356 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.903630788000001, 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.357 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.358 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.359 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:26.360 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.361 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.362 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.901065547999998, 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.366 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.367 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.368 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.369 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:26.369 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:26.369 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.369 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.369 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.370 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:26.371 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:26.372 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.372 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.372 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:26.373 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:26.374 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.900104055, 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.375 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.376 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.377 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.378 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.898021495999998, 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.381 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.382 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:26.383 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:26.384 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:26.385 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.896990159000001, 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.386 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.387 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.388 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:26.389 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.896059610000002, 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.390 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.391 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:26.392 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.393 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:26.394 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.895204201999999, 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.395 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.396 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:26.397 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.398 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.399 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:26.400 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:26.401 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.892687780999999, 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.402 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.403 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:26.404 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.405 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.406 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.340648069, 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:26.407 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.408 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:26.409 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.410 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:26.411 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.339233844999999, 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:26.412 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.413 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:26.414 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:26.415 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.338227823, 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.416 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:26.417 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:26.418 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:26.419 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 31.336957195, 2024-09-15T01:44:26.420 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.421 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:26.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.423 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.424 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 30.781259115000001, 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.425 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:26.426 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.427 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.428 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=3, to_read={3:720ccbad:::smithi03327189-29:head=0,262144,0}, subchunks={3:720ccbad:::smithi03327189-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.809501273, 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033057709999999999, 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.429 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.430 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537073+0000", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2750000000000001e-06 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537091+0000", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537118+0000", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7220999999999999e-05 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.431 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537130+0000", 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1551e-05 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537215+0000", 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.5156999999999998e-05 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.540378+0000", 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0031629000000000002 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.432 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=7, to_read={3:7de15fb4:::smithi03327189-49:head=0,262144,0}, subchunks={3:7de15fb4:::smithi03327189-49:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.801267854000001, 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033061729999999999, 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.433 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545307+0000", 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0699999999999999e-06 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.434 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545320+0000", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2867000000000001e-05 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545339+0000", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9352e-05 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:26.435 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8869999999999998e-06 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545406+0000", 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7782999999999999e-05 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548612+0000", 2024-09-15T01:44:26.436 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032062140000000002 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.7s1 33/32 ECSubRead(tid=46, to_read={3:e02ccbe4:::smithi03327189-36:head=0,262144,0}, subchunks={3:e02ccbe4:::smithi03327189-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.795401403, 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033818339999999998, 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.437 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551173+0000", 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.111e-06 2024-09-15T01:44:26.438 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551185+0000", 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.205e-05 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551202+0000", 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6971999999999999e-05 2024-09-15T01:44:26.439 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551209+0000", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5339999999999996e-06 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551283+0000", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.4664000000000002e-05 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554554+0000", 2024-09-15T01:44:26.440 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032705030000000001 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=67, to_read={3:ac2d3331:::smithi03327189-18:head=0,262144,0}, subchunks={3:ac2d3331:::smithi03327189-18:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.793449659, 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003506547, 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.441 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553125+0000", 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2900000000000002e-07 2024-09-15T01:44:26.442 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553136+0000", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1032999999999999e-05 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553161+0000", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5043000000000001e-05 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553170+0000", 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8729999999999996e-06 2024-09-15T01:44:26.443 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553235+0000", 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5230999999999999e-05 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556630+0000", 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003395538 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.444 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.cs7 33/32 ECSubRead(tid=68, to_read={3:3584cd82:::smithi03327189-23:head=0,262144,0}, subchunks={3:3584cd82:::smithi03327189-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.793386212, 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036502639999999999, 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.445 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553188+0000", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2599999999999998e-07 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553194+0000", 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3770000000000001e-06 2024-09-15T01:44:26.446 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553204+0000", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553210+0000", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.8850000000000004e-06 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553352+0000", 2024-09-15T01:44:26.447 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014202600000000001 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556838+0000", 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003485868 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=71, to_read={3:a22eae17:::smithi03327189-26:head=0,262144,0}, subchunks={3:a22eae17:::smithi03327189-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:26.448 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 13.781083122, 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0030758130000000002, 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.449 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565491+0000", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2699999999999998e-07 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565507+0000", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6076e-05 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565523+0000", 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:44:26.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565529+0000", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565605+0000", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6344e-05 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.568566+0000", 2024-09-15T01:44:26.451 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00296131 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:26.452 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:26.453 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:26.447+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:26.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:26.508 INFO:tasks.rados.rados.0.smithi033.stdout:150: done (2 left) 2024-09-15T01:44:26.508 INFO:tasks.rados.rados.0.smithi033.stdout:154: done (1 left) 2024-09-15T01:44:26.508 INFO:tasks.rados.rados.0.smithi033.stdout:156: done (0 left) 2024-09-15T01:44:26.508 INFO:tasks.rados.rados.0.smithi033.stdout:157: copy_from oid 13 from oid 6 current snap is 9 2024-09-15T01:44:26.508 INFO:tasks.rados.rados.0.smithi033.stdout:158: copy_from oid 38 from oid 47 current snap is 9 2024-09-15T01:44:26.509 INFO:tasks.rados.rados.0.smithi033.stdout:159: snap_create 2024-09-15T01:44:26.516 INFO:tasks.rados.rados.0.smithi033.stdout:158: finishing copy_from to smithi03327189-38 2024-09-15T01:44:26.516 INFO:tasks.rados.rados.0.smithi033.stdout:158: got expected ENOENT (src dne) 2024-09-15T01:44:26.516 INFO:tasks.rados.rados.0.smithi033.stdout:158: finishing copy_from racing read to smithi03327189-38 2024-09-15T01:44:26.520 INFO:tasks.rados.rados.0.smithi033.stdout:157: finishing copy_from racing read to smithi03327189-13 2024-09-15T01:44:26.524 INFO:tasks.rados.rados.0.smithi033.stdout:157: finishing copy_from to smithi03327189-13 2024-09-15T01:44:26.525 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 13 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:26.554 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:26.547+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:26.567 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:26.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:26.567 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:26.567 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:26.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:26.655 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:26.648+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:26.672 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:26.673 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:26.673 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:26.673 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:26.673 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:26.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:26.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:26.752+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:26.786 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.253570072, 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.251138674, 2024-09-15T01:44:26.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:26.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.250899859, 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:26.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.250868476000001, 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:26.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.271661046, 2024-09-15T01:44:26.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.263746643999999, 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.821 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.263717380999999, 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:26.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.236280535000001, 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:26.834 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.23511379, 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.234740517000001, 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:26.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:26.842 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.234021418999999, 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:26.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.2518068489999994, 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:26.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:26.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2641420620000003, 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:26.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2619036010000002, 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:26.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2613339320000003, 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:26.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2612749640000001, 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:26.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=13, reqid=client.4632.0:302, at_version=43'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:26.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2596820859999998, 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094416429999999996, 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520336+0000", 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520377+0000", 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1192000000000001e-05 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520398+0000", 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1013e-05 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520411+0000", 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.239e-05 2024-09-15T01:44:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522362+0000", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001951803 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522402+0000", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9764000000000001e-05 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529721+0000", 2024-09-15T01:44:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007318504 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529777+0000", 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6501e-05 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:26.882 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2459127969999999, 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083306100000000004, 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534128+0000", 2024-09-15T01:44:26.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.315e-05 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534140+0000", 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1508999999999999e-05 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534146+0000", 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3230000000000003e-06 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534199+0000", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3056999999999997e-05 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534209+0000", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7999999999999993e-06 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542395+0000", 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081854519999999993 2024-09-15T01:44:26.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542435+0000", 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0759000000000001e-05 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.273376756, 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:26.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:26.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.893 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.27328297800000001, 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:26.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:26.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:26.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:26.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:26.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:26.900 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:26.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:26.902 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:26.899+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:27.002 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:27.003 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.003 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:26.996+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:27.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:27.104 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:27.096+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:27.113 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.113 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.114 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:27.114 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:27.114 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:27.205 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:27.200+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:27.229 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.702841105999999, 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:27.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.702802101, 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:27.238 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.702771157000001, 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=63, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.689292347, 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058134440000000001, 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.13e-07 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534542+0000", 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3890000000000002e-06 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534557+0000", 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4919000000000001e-05 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534562+0000", 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7189999999999998e-06 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540154+0000", 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055920869999999999 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540348+0000", 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019381699999999999 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=64, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.689259061, 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059416290000000004, 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534573+0000", 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534579+0000", 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6479999999999997e-06 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534584+0000", 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6079999999999998e-06 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540397+0000", 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058133880000000001 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540510+0000", 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001125 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.250 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.688380038, 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.707886857, 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.255 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:27.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:27.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=59, reqid=osd.3.0:59, at_version=38'16, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.69444208, 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062207900000000003, 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529386+0000", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1200000000000002e-07 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529410+0000", 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4343000000000001e-05 2024-09-15T01:44:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529422+0000", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2128e-05 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529429+0000", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7750000000000004e-06 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529473+0000", 2024-09-15T01:44:27.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3523000000000003e-05 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529479+0000", 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3720000000000001e-06 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535566+0000", 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006087528 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535606+0000", 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9409000000000002e-05 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=60, reqid=osd.3.0:60, at_version=38'18, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6916656680000006, 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060140539999999996, 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:27.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8100000000000001e-07 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532195+0000", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2177e-05 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532208+0000", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2991e-05 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532217+0000", 2024-09-15T01:44:27.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8309999999999996e-06 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532269+0000", 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080000000000003e-05 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532279+0000", 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6260000000000007e-06 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538153+0000", 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058734499999999997 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538176+0000", 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3017999999999999e-05 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.7058913360000005, 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:27.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:27.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:27.274 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.7058218039999993, 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.7057992679999998, 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:27.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:27.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6955044729999997, 2024-09-15T01:44:27.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071464149999999997, 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.06e-07 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528345+0000", 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1920999999999999e-05 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528355+0000", 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4830000000000005e-06 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528360+0000", 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5029999999999997e-06 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528430+0000", 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9640000000000004e-05 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528440+0000", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0611999999999999e-05 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535446+0000", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007005173 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535469+0000", 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3577000000000001e-05 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7057320760000003, 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:27.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:27.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7056625509999996, 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:27.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:27.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:27.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7054204730000002, 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:27.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:27.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7053670390000004, 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:27.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:27.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.314 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7052948600000004, 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:27.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6913333020000003, 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:27.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.4s0 43/34 ECSubReadReply(tid=50, attrs_read=0))", 2024-09-15T01:44:27.325 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6897867929999997, 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063210219999999999, 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534041+0000", 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.37e-07 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537228+0000", 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031869820000000001 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537240+0000", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2371999999999999e-05 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537247+0000", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8820000000000003e-06 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.539382+0000", 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021353000000000001 2024-09-15T01:44:27.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540361+0000", 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097894900000000001 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.329 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:27.331 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:27.327+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:27.432 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:27.423+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:27.435 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:27.435 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:27.506 INFO:tasks.rados.rados.0.smithi033.stdout:157: done (2 left) 2024-09-15T01:44:27.507 INFO:tasks.rados.rados.0.smithi033.stdout:158: done (1 left) 2024-09-15T01:44:27.507 INFO:tasks.rados.rados.0.smithi033.stdout:159: done (0 left) 2024-09-15T01:44:27.507 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 18 current snap is 10 2024-09-15T01:44:27.507 INFO:tasks.rados.rados.0.smithi033.stdout:160: seq_num 85 ranges {294912=671744,966656=770048} 2024-09-15T01:44:27.518 INFO:tasks.rados.rados.0.smithi033.stdout:160: writing smithi03327189-18 from 294912 to 966656 tid 1 2024-09-15T01:44:27.524 INFO:tasks.rados.rados.0.smithi033.stdout:160: writing smithi03327189-18 from 966656 to 1736704 tid 2 2024-09-15T01:44:27.525 INFO:tasks.rados.rados.0.smithi033.stdout:161: read oid 1 snap -1 2024-09-15T01:44:27.525 INFO:tasks.rados.rados.0.smithi033.stdout:161: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:44:27.525 INFO:tasks.rados.rados.0.smithi033.stdout:162: snap_remove snap 8 2024-09-15T01:44:27.532 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:27.524+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:27.547 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.547 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.547 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:27.547 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:27.547 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:27.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:44:27.633 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:27.628+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.126567965, 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003107659, 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.657 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526021+0000", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526022+0000", 2024-09-15T01:44:27.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899999999999999e-06 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526051+0000", 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8187e-05 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526061+0000", 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0343000000000001e-05 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526068+0000", 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7469999999999999e-06 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526304+0000", 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000236197 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526315+0000", 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0946e-05 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529101+0000", 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027858359999999999 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.529129+0000", 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8313000000000001e-05 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.117097215999999, 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:27.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:27.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.136678140000001, 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040598090000000002, 2024-09-15T01:44:27.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515912+0000", 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0800000000000005e-07 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515965+0000", 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3362999999999997e-05 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515984+0000", 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9130000000000001e-05 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516042+0000", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7518000000000001e-05 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516184+0000", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014263599999999999 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516230+0000", 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5240000000000001e-05 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519914+0000", 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036847939999999999 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519971+0000", 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6619999999999997e-05 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.133332246, 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:27.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:27.675 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.133181318, 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:27.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:27.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.13057888, 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050197059999999996, 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522012+0000", 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.122e-06 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522045+0000", 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3034999999999998e-05 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522067+0000", 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2742e-05 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522078+0000", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0278000000000001e-05 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524135+0000", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020572450000000001 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524153+0000", 2024-09-15T01:44:27.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8510000000000001e-05 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527017+0000", 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028639719999999998 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527030+0000", 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2802e-05 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=97, reqid=osd.5.0:95, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.123864684999999, 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041007480000000004, 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528726+0000", 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8700000000000004e-07 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528739+0000", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3694e-05 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528757+0000", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501000000000001e-05 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528763+0000", 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7270000000000001e-06 2024-09-15T01:44:27.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528985+0000", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000221955 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528996+0000", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0363000000000001e-05 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532796+0000", 2024-09-15T01:44:27.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003800257 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532825+0000", 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9363999999999999e-05 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1308467219999994, 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.690 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:27.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=34, reqid=client.4632.0:252, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1283234269999998, 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056439819999999996, 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524309+0000", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2670000000000003e-05 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524323+0000", 2024-09-15T01:44:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3813e-05 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524331+0000", 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2300000000000008e-06 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524482+0000", 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015112700000000001 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524497+0000", 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4666e-05 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529866+0000", 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053691579999999997 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529910+0000", 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3990999999999999e-05 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=35, reqid=client.4632.0:253, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1263962289999991, 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038225540000000001, 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526194+0000", 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9099999999999991e-07 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526216+0000", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2107e-05 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526228+0000", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.169e-05 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526235+0000", 2024-09-15T01:44:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7309999999999996e-06 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526305+0000", 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0108e-05 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526316+0000", 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1347e-05 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529991+0000", 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036753570000000002 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530016+0000", 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4223000000000002e-05 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.4s5 43/34 ECSubRead(tid=48, to_read={3:2cc5cf52:::smithi03327189-46:head=0,421888,0}, subchunks={3:2cc5cf52:::smithi03327189-46:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1256827229999997, 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029064889999999999, 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526907+0000", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526914+0000", 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1659999999999997e-06 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526926+0000", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2221e-05 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526934+0000", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3459999999999996e-06 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527012+0000", 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8615000000000005e-05 2024-09-15T01:44:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529813+0000", 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028007430000000001 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.4s5 43/34 ECSubRead(tid=49, to_read={3:2cc5cf52:::smithi03327189-46:head=0,421888,0}, subchunks={3:2cc5cf52:::smithi03327189-46:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1256409610000002, 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00302615, 2024-09-15T01:44:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526948+0000", 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526949+0000", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9299999999999999e-07 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526955+0000", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7769999999999997e-06 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526964+0000", 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5720000000000003e-06 2024-09-15T01:44:27.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526970+0000", 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6380000000000001e-06 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529855+0000", 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028850289999999999 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529975+0000", 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011984100000000001 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.7s1 43/34 ECSubRead(tid=73, to_read={3:e5da12c9:::smithi03327189-25:head=0,102400,0}, subchunks={3:e5da12c9:::smithi03327189-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1235445879999997, 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035108980000000001, 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6800000000000001e-07 2024-09-15T01:44:27.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529051+0000", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1720000000000004e-06 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529064+0000", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2435999999999999e-05 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529069+0000", 2024-09-15T01:44:27.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2009999999999998e-06 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532343+0000", 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032738820000000001 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532556+0000", 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021283900000000001 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s4 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1197452999999999, 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046971579999999999, 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532845+0000", 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5600000000000005e-07 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532856+0000", 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1635e-05 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532868+0000", 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1712000000000001e-05 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532874+0000", 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3579999999999999e-06 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533006+0000", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013125800000000001 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533016+0000", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537501+0000", 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044853799999999997 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537541+0000", 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790999999999999e-05 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1192644109999996, 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042471389999999996, 2024-09-15T01:44:27.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533350+0000", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.427e-05 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533361+0000", 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1258e-05 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533367+0000", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.438e-06 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533417+0000", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9325e-05 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533427+0000", 2024-09-15T01:44:27.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0468999999999999e-05 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537550+0000", 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041229780000000002 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537572+0000", 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2106000000000001e-05 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=135, reqid=client.4632.0:304, at_version=43'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1035552239999999, 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059241329999999998, 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549410+0000", 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000375464 2024-09-15T01:44:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549431+0000", 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713999999999999e-05 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549438+0000", 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549499+0000", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1303999999999994e-05 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549514+0000", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4901e-05 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554929+0000", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054145510000000001 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554958+0000", 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9393000000000001e-05 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/34 ECSubWrite(tid=54, reqid=client.4632.0:294, at_version=0'0, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1011082270000001, 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032223550000000001, 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.730 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551482+0000", 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551498+0000", 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6297e-05 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551514+0000", 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5673999999999999e-05 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551522+0000", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6969999999999993e-06 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551573+0000", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1700999999999998e-05 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551584+0000", 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-15T01:44:27.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554677+0000", 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030922670000000001 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554703+0000", 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6950000000000001e-05 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=136, reqid=client.4632.0:305, at_version=43'13, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.099078606, 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043787469999999997, 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553519+0000", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2460000000000003e-06 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554227+0000", 2024-09-15T01:44:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00070824899999999999 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554241+0000", 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4096e-05 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554249+0000", 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554324+0000", 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5050999999999997e-05 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554335+0000", 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0721e-05 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557866+0000", 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003531621 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557890+0000", 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3088999999999999e-05 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=137, reqid=client.4632.0:306, at_version=43'14, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0974519750000002, 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046968319999999997, 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555138+0000", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8599999999999998e-07 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555169+0000", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0957000000000002e-05 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555181+0000", 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1789e-05 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555188+0000", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7479999999999996e-06 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555224+0000", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5938e-05 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555230+0000", 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.998e-06 2024-09-15T01:44:27.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559806+0000", 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045767990000000003 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559834+0000", 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7917000000000001e-05 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=138, reqid=client.4632.0:307, at_version=43'15, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0959204869999999, 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031781679999999999, 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556670+0000", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556689+0000", 2024-09-15T01:44:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8981999999999999e-05 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556706+0000", 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7272999999999999e-05 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556713+0000", 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8279999999999997e-06 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556755+0000", 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2341e-05 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556764+0000", 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8300000000000002e-06 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559837+0000", 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030732849999999998 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:27.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559847+0000", 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:27.748 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:27.750 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:27.743+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:27.846 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.846 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.846 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:27.846 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.850 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:27.848+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:27.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:27.951 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:27.944+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:27.955 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:27.956 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:27.965 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:28.051 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:28.044+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.550229541, 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.550175302, 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.550156164000001, 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:206 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~606208] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.539837351999999, 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062804979999999998, 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 206 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519793+0000", 2024-09-15T01:44:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0190000000000001e-06 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520167+0000", 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037447000000000001 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520180+0000", 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3101e-05 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520186+0000", 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9989999999999997e-06 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521198+0000", 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001011807 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521308+0000", 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110059 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522439+0000", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011307229999999999 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525693+0000", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003254159 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526006+0000", 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000312971 2024-09-15T01:44:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526071+0000", 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5190000000000004e-05 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:207 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.538134419999999, 2024-09-15T01:44:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055629219999999997, 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 207 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521495+0000", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.618e-06 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521721+0000", 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022585199999999999 2024-09-15T01:44:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521741+0000", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0496000000000002e-05 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521749+0000", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2620000000000001e-06 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523331+0000", 2024-09-15T01:44:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015822919999999999 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523435+0000", 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001044 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524251+0000", 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000815367 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526200+0000", 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019496120000000001 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526995+0000", 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079458399999999998 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527056+0000", 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1439000000000006e-05 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:208 3.2s0 3:47035a48:::smithi03327189-20:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.537868579000001, 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056281209999999998, 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 208 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521760+0000", 2024-09-15T01:44:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521765+0000", 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521774+0000", 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8240000000000006e-06 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521778+0000", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3760000000000001e-06 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524887+0000", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031084820000000001 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525036+0000", 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000149736 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525431+0000", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039431399999999999 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527107+0000", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001676147 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527310+0000", 2024-09-15T01:44:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000203632 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527387+0000", 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6938000000000001e-05 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:209 3.2s0 3:47035a48:::smithi03327189-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.537845700999998, 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079111819999999992, 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 209 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521783+0000", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521784+0000", 2024-09-15T01:44:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.736e-06 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521791+0000", 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0849999999999999e-06 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521803+0000", 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526224+0000", 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004421431 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526350+0000", 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012618500000000001 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527766+0000", 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014151299999999999 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527853+0000", 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7435999999999994e-05 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529693+0000", 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001840436 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=61, reqid=client.4632.0:199, at_version=33'22, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.534353695, 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005452106, 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525274+0000", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3999999999999998e-07 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525306+0000", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1396000000000002e-05 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525315+0000", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7180000000000002e-06 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525321+0000", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.229e-06 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525509+0000", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018865900000000001 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.525520+0000", 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0285000000000001e-05 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530704+0000", 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051837840000000003 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.530726+0000", 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2795000000000001e-05 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0, backfill_or_async_recovery))", 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.527453151, 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083125319999999992, 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532176+0000", 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:44:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2191000000000001e-05 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6229999999999999e-06 2024-09-15T01:44:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532292+0000", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3135999999999999e-05 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532304+0000", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2481e-05 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540422+0000", 2024-09-15T01:44:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081175480000000005 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540487+0000", 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5278999999999996e-05 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.524354043000001, 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077245059999999999, 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535275+0000", 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000001e-07 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535296+0000", 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1590000000000002e-05 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535309+0000", 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2247e-05 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535315+0000", 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7870000000000004e-06 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535399+0000", 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3398000000000003e-05 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535410+0000", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542978+0000", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075673989999999998 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542998+0000", 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0469999999999999e-05 2024-09-15T01:44:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4, backfill_or_async_recovery))", 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.522789606, 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061654960000000003, 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536839+0000", 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536859+0000", 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0585999999999999e-05 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536869+0000", 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3989999999999993e-06 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536878+0000", 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6180000000000001e-06 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536929+0000", 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0285999999999999e-05 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536939+0000", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0472000000000001e-05 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542974+0000", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006034569 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543004+0000", 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0196999999999999e-05 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.538976866, 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065983789999999997, 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520652+0000", 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0330000000000001e-06 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520685+0000", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3210999999999998e-05 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520702+0000", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6880000000000001e-05 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520712+0000", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0003000000000001e-05 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520819+0000", 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106947 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520835+0000", 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5647000000000001e-05 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527222+0000", 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063877339999999999 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527249+0000", 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6923999999999998e-05 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 38/34 ECSubWrite(tid=42, reqid=osd.0.0:42, at_version=38'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.535014478000001, 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060356519999999999, 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524614+0000", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7999999999999996e-07 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524667+0000", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2803000000000002e-05 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524683+0000", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6623e-05 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524692+0000", 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5169999999999999e-06 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524799+0000", 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010671300000000001 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524810+0000", 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1661999999999999e-05 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530622+0000", 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058122499999999997 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530649+0000", 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6604000000000001e-05 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es6 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5324821570000005, 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052142990000000004, 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527146+0000", 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5699999999999998e-07 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527172+0000", 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6142000000000001e-05 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527191+0000", 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8192000000000001e-05 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527198+0000", 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527308+0000", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109604 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527319+0000", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1124999999999999e-05 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.532330+0000", 2024-09-15T01:44:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050108569999999996 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.532360+0000", 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0133000000000001e-05 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.5411812640000004, 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.5411136020000002, 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.5410878610000003, 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.528017052, 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.5269751469999999, 2024-09-15T01:44:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.5261543419999999, 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:28.162 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:28.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:28.164 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:28.164 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:28.160+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:28.264 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:28.259+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:28.275 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:28.276 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:28.276 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:28.276 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:28.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:28.354 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:28.365 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:28.363+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:28.377 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:28.385 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:28.385 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:28.385 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:28.385 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:28.385 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:28.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:28.395 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:28.415 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:28.436 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:28.457 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:28.466 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:28.460+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:28.476 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:28.496 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:28.496 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:28.498 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:28.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:28.512 INFO:tasks.rados.rados.0.smithi033.stdout:162: done (2 left) 2024-09-15T01:44:28.512 INFO:tasks.rados.rados.0.smithi033.stdout:163: rmattr oid 30 current snap is 10 2024-09-15T01:44:28.512 INFO:tasks.rados.rados.0.smithi033.stdout:163: done (2 left) 2024-09-15T01:44:28.512 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 47 current snap is 10 2024-09-15T01:44:28.512 INFO:tasks.rados.rados.0.smithi033.stdout:164: seq_num 86 ranges {0=688128,688128=540672,1228800=229376} 2024-09-15T01:44:28.514 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:44:28.520 INFO:tasks.rados.rados.0.smithi033.stdout:164: writing smithi03327189-47 from 0 to 688128 tid 1 2024-09-15T01:44:28.523 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:28.523 INFO:tasks.rados.rados.0.smithi033.stdout:164: writing smithi03327189-47 from 688128 to 1228800 tid 2 2024-09-15T01:44:28.525 INFO:tasks.rados.rados.0.smithi033.stdout:164: writing smithi03327189-47 from 1228800 to 1458176 tid 3 2024-09-15T01:44:28.525 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 9 current snap is 10 2024-09-15T01:44:28.525 INFO:tasks.rados.rados.0.smithi033.stdout:161: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:44:28.526 INFO:tasks.rados.rados.0.smithi033.stdout:160: finishing write tid 1 to smithi03327189-18 2024-09-15T01:44:28.526 INFO:tasks.rados.rados.0.smithi033.stdout:160: finishing write tid 2 to smithi03327189-18 2024-09-15T01:44:28.526 INFO:tasks.rados.rados.0.smithi033.stdout:160: finishing write tid 3 to smithi03327189-18 2024-09-15T01:44:28.526 INFO:tasks.rados.rados.0.smithi033.stdout:165: seq_num 87 ranges {1490944=704512,2195456=688128,2883584=81920} 2024-09-15T01:44:28.544 INFO:tasks.rados.rados.0.smithi033.stdout:165: writing smithi03327189-9 from 1490944 to 2195456 tid 1 2024-09-15T01:44:28.546 INFO:tasks.rados.rados.0.smithi033.stdout:165: writing smithi03327189-9 from 2195456 to 2883584 tid 2 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:165: writing smithi03327189-9 from 2883584 to 2965504 tid 3 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:161: done (3 left) 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 11 current snap is 10 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:160: finishing write tid 4 to smithi03327189-18 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 59 (ObjNum 85 snap 10 seq_num 85) dirty exists 2024-09-15T01:44:28.547 INFO:tasks.rados.rados.0.smithi033.stdout:160: left oid 18 (ObjNum 85 snap 10 seq_num 85) 2024-09-15T01:44:28.548 INFO:tasks.rados.rados.0.smithi033.stdout:166: seq_num 88 ranges {1179648=720896,1900544=442368,2342912=327680} 2024-09-15T01:44:28.556 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:28.559 INFO:tasks.rados.rados.0.smithi033.stdout:166: writing smithi03327189-11 from 1179648 to 1900544 tid 1 2024-09-15T01:44:28.562 INFO:tasks.rados.rados.0.smithi033.stdout:166: writing smithi03327189-11 from 1900544 to 2342912 tid 2 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:166: writing smithi03327189-11 from 2342912 to 2670592 tid 3 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:160: done (3 left) 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:167: delete oid 45 current snap is 10 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:164: finishing write tid 1 to smithi03327189-47 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:164: finishing write tid 2 to smithi03327189-47 2024-09-15T01:44:28.565 INFO:tasks.rados.rados.0.smithi033.stdout:164: finishing write tid 3 to smithi03327189-47 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:164: finishing write tid 4 to smithi03327189-47 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:164: finishing write tid 5 to smithi03327189-47 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 63 (ObjNum 86 snap 10 seq_num 86) dirty exists 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:164: left oid 47 (ObjNum 86 snap 10 seq_num 86) 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: finishing write tid 1 to smithi03327189-9 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: finishing write tid 2 to smithi03327189-9 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: finishing write tid 3 to smithi03327189-9 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: finishing write tid 4 to smithi03327189-9 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: finishing write tid 5 to smithi03327189-9 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 9 v 39 (ObjNum 87 snap 10 seq_num 87) dirty exists 2024-09-15T01:44:28.566 INFO:tasks.rados.rados.0.smithi033.stdout:165: left oid 9 (ObjNum 87 snap 10 seq_num 87) 2024-09-15T01:44:28.568 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:28.560+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:28.569 INFO:tasks.rados.rados.0.smithi033.stdout:166: finishing write tid 1 to smithi03327189-11 2024-09-15T01:44:28.571 INFO:tasks.rados.rados.0.smithi033.stdout:166: finishing write tid 2 to smithi03327189-11 2024-09-15T01:44:28.571 INFO:tasks.rados.rados.0.smithi033.stdout:164: done (3 left) 2024-09-15T01:44:28.571 INFO:tasks.rados.rados.0.smithi033.stdout:165: done (2 left) 2024-09-15T01:44:28.571 INFO:tasks.rados.rados.0.smithi033.stdout:167: done (1 left) 2024-09-15T01:44:28.572 INFO:tasks.rados.rados.0.smithi033.stdout:168: snap_create 2024-09-15T01:44:28.572 INFO:tasks.rados.rados.0.smithi033.stdout:166: finishing write tid 3 to smithi03327189-11 2024-09-15T01:44:28.573 INFO:tasks.rados.rados.0.smithi033.stdout:166: finishing write tid 4 to smithi03327189-11 2024-09-15T01:44:28.575 INFO:tasks.rados.rados.0.smithi033.stdout:166: finishing write tid 5 to smithi03327189-11 2024-09-15T01:44:28.575 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 32 (ObjNum 88 snap 10 seq_num 88) dirty exists 2024-09-15T01:44:28.575 INFO:tasks.rados.rados.0.smithi033.stdout:166: left oid 11 (ObjNum 88 snap 10 seq_num 88) 2024-09-15T01:44:28.580 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:28.583 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] out_osds: [11] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:28.583 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:28.584 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] 2024-09-15T01:44:28.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-15T01:44:28.602 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:28.605 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:28.617 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:28.617 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:28.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:28.668 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:28.663+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:28.769 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:28.763+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:28.774 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:28.774 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:28.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:28.870 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:28.864+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:28.883 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:28.883 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:28.884 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-09-15T01:44:28.880+0000 7fed2b4c9700 -1 mon.a@0(leader).osd e49 definitely_dead 0 2024-09-15T01:44:28.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:28.970 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:28.968+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:28.991 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:28.991 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:29.072 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:29.068+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:29.104 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.104 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:29.173 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:29.168+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:29.213 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.213 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:29.274 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:29.267+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:29.324 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.324 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:29.375 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:29.368+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:29.434 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.435 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:29.475 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:29.468+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:29.526 INFO:tasks.rados.rados.0.smithi033.stdout:166: done (1 left) 2024-09-15T01:44:29.527 INFO:tasks.rados.rados.0.smithi033.stdout:168: done (0 left) 2024-09-15T01:44:29.527 INFO:tasks.rados.rados.0.smithi033.stdout:169: read oid 31 snap -1 2024-09-15T01:44:29.527 INFO:tasks.rados.rados.0.smithi033.stdout:169: expect (ObjNum 51 snap 0 seq_num 51) 2024-09-15T01:44:29.527 INFO:tasks.rados.rados.0.smithi033.stdout:170: delete oid 48 current snap is 11 2024-09-15T01:44:29.534 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:29.528+0000 7fb3bafd6700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:44:29.535 INFO:teuthology.orchestra.run.smithi033.stderr:marked out osd.5. 2024-09-15T01:44:29.576 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:29.567+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:29.600 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.600 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:29.677 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:29.671+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:29.725 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.725 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:29.778 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:29.772+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:29.833 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.833 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.842 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:29.879 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:29.871+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:29.939 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:29.939 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:29.948 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:29.979 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:29.972+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:30.080 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:30.076+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:30.093 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.093 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:30.181 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:30.176+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:30.200 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.200 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.209 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:30.282 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:30.276+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:30.309 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.309 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.318 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:30.383 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:30.376+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:30.416 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.417 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.425 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:30.484 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:30.480+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:30.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:44:30.530 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:30.523+0000 7f84b3625700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:44:30.532 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.532 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.542 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:30.585 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:30.579+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:30.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:30.631 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.632 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:30.685 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:30.680+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:30.748 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:30.775 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:30.786 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:30.780+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:30.787 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.787 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.796 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:30.818 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:30.840 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:30.863 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:30.887 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:30.879+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:30.897 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:30.898 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:30.907 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:30.909 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:30.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:30.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:30.983 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:30.988 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:30.984+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:31.006 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "size": 20, 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 600, 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "ops": [ 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:40 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'22, mlcod=19'19)", 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:31.054 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.607964185, 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003540844, 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.055 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441981+0000", 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441982+0000", 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.248e-06 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.441992+0000", 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8830000000000001e-06 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.056 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442008+0000", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5591e-05 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442029+0000", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0781999999999999e-05 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442159+0000", 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00013060799999999999 2024-09-15T01:44:31.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442193+0000", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.3429000000000001e-05 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445451+0000", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032581789999999999 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445522+0000", 2024-09-15T01:44:31.058 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1124000000000005e-05 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:41 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'23, mlcod=19'19)", 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.607002657999999, 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033974589999999998, 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.059 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442943+0000", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.442944+0000", 2024-09-15T01:44:31.060 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3659999999999999e-06 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443189+0000", 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00024533600000000002 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443215+0000", 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5156000000000001e-05 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.061 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443224+0000", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8460000000000003e-06 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443322+0000", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7305000000000004e-05 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.443362+0000", 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0330000000000002e-05 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.062 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446274+0000", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0029120330000000001 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446340+0000", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.6087000000000006e-05 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:44 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'26, mlcod=20'20)", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:31.063 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.604437418000003, 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0037702, 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445508+0000", 2024-09-15T01:44:31.064 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445509+0000", 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4869999999999999e-06 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445623+0000", 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000113967 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.065 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445652+0000", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8943000000000001e-05 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445670+0000", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7693000000000002e-05 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445743+0000", 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2791000000000002e-05 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.066 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.445775+0000", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2073000000000003e-05 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449207+0000", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003431824 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449278+0000", 2024-09-15T01:44:31.067 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.1421999999999995e-05 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:45 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'27, mlcod=20'20)", 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.603475924999998, 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0054465709999999999, 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.068 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446469+0000", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446471+0000", 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.3790000000000001e-06 2024-09-15T01:44:31.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446485+0000", 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.4243000000000001e-05 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446514+0000", 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9127000000000001e-05 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.070 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446546+0000", 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1681999999999999e-05 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446614+0000", 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.8532e-05 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.446646+0000", 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1266999999999997e-05 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451862+0000", 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0052168309999999999 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451916+0000", 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.3510000000000001e-05 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.072 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:46 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'28, mlcod=20'24)", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.601393366000003, 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0058851270000000004, 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.073 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448552+0000", 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448553+0000", 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.39e-06 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448566+0000", 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.306e-05 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.074 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448597+0000", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1028999999999997e-05 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448616+0000", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.8723999999999999e-05 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448772+0000", 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000156119 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.075 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.448816+0000", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3739000000000003e-05 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454357+0000", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055406379999999996 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454437+0000", 2024-09-15T01:44:31.076 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.0427999999999996e-05 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:47 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'29, mlcod=20'24)", 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.600362029000003, 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0049142659999999996, 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.077 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449583+0000", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449585+0000", 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5400000000000001e-06 2024-09-15T01:44:31.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449714+0000", 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000129114 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449746+0000", 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2113000000000002e-05 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.079 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449758+0000", 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1616e-05 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449869+0000", 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000111525 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.449935+0000", 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5308000000000006e-05 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.080 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454441+0000", 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0045066020000000002 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454498+0000", 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.6447999999999999e-05 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:48 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'30, mlcod=20'24)", 2024-09-15T01:44:31.081 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.59943148, 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0040436209999999998, 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450514+0000", 2024-09-15T01:44:31.082 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450515+0000", 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.53e-06 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450542+0000", 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.6693e-05 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.083 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450586+0000", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.3433000000000002e-05 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450596+0000", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0923e-05 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450660+0000", 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3452000000000002e-05 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.084 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.450688+0000", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8282000000000001e-05 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454500+0000", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0038121269999999998 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454557+0000", 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7181e-05 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.085 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:49 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'31, mlcod=20'25)", 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.598576072, 2024-09-15T01:44:31.086 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003567494, 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451369+0000", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451370+0000", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0699999999999996e-07 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451380+0000", 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.4099999999999997e-06 2024-09-15T01:44:31.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451403+0000", 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.2912e-05 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451413+0000", 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0047000000000001e-05 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.090 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451499+0000", 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6260000000000004e-05 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.451538+0000", 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.8911000000000001e-05 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454888+0000", 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033498009999999999 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.091 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454937+0000", 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.9246000000000003e-05 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:52 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'34, mlcod=20'26)", 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.596059650999997, 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003825887, 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.092 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453886+0000", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.093 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453887+0000", 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.606e-06 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453900+0000", 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.237e-05 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453931+0000", 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.1439999999999997e-05 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.094 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.453947+0000", 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5979000000000001e-05 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454034+0000", 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.6805999999999995e-05 2024-09-15T01:44:31.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.454064+0000", 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0277000000000002e-05 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457629+0000", 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0035643799999999998 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.096 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:54.457712+0000", 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.3028999999999994e-05 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:72 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'50, mlcod=20'46)", 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.044019939000002, 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036300310000000001, 2024-09-15T01:44:31.097 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005925+0000", 2024-09-15T01:44:31.098 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.005926+0000", 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2999999999999999e-07 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006018+0000", 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.1589999999999996e-05 2024-09-15T01:44:31.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006038+0000", 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0043000000000001e-05 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006051+0000", 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2862e-05 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.100 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006134+0000", 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2676000000000006e-05 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.006166+0000", 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2561000000000002e-05 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.101 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009454+0000", 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003288221 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009555+0000", 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000101148 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.102 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:73 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'51, mlcod=20'47)", 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.042605715000001, 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0055863400000000004, 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.103 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007340+0000", 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007341+0000", 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0499999999999999e-06 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.104 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007350+0000", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.0140000000000004e-06 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007370+0000", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0571999999999999e-05 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007380+0000", 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.3850000000000008e-06 2024-09-15T01:44:31.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007452+0000", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.2622999999999999e-05 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.007481+0000", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.8813e-05 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012871+0000", 2024-09-15T01:44:31.106 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.005389853 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012926+0000", 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.503e-05 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:74 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=20'47)", 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.041599693000002, 2024-09-15T01:44:31.107 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0046123709999999997, 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008346+0000", 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.108 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008347+0000", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.8200000000000008e-07 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008356+0000", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.7049999999999994e-06 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008375+0000", 2024-09-15T01:44:31.109 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9110000000000002e-05 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008383+0000", 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3679999999999996e-06 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008469+0000", 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.666e-05 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.110 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.008500+0000", 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.0941000000000001e-05 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012929+0000", 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0044285280000000002 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.012958+0000", 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.9077e-05 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.111 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:75 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=20'48)", 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 36.040329065000002, 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0034470289999999999, 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.112 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.113 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.113 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.113 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.113 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:31.113 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009616+0000", 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.114 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009617+0000", 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.06e-06 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.009991+0000", 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000373295 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.115 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010011+0000", 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0747999999999999e-05 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010020+0000", 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1259999999999998e-06 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010093+0000", 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.3706000000000002e-05 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.116 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.010125+0000", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 3.2183e-05 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013012+0000", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0028867519999999998 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.013063+0000", 2024-09-15T01:44:31.117 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.1159000000000003e-05 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "osd_repop(client.4204.0:102 1.0 e21/18 1:30306672:devicehealth::main.db.0000000000000000:head v 21'76, mlcod=21'73)", 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 35.484630985000003, 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003818101, 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.118 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565314+0000", 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.119 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.565315+0000", 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.71e-07 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566255+0000", 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00093947599999999996 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566275+0000", 2024-09-15T01:44:31.120 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.0151000000000001e-05 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566283+0000", 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.1459999999999996e-06 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566403+0000", 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.000119484 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "started", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.566443+0000", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.0905000000000002e-05 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "commit_sent", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569053+0000", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0026093900000000001 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:43:55.569132+0000", 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.9678000000000005e-05 2024-09-15T01:44:31.122 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=3, to_read={3:720ccbad:::smithi03327189-29:head=0,262144,0}, subchunks={3:720ccbad:::smithi03327189-29:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.512873143, 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033057709999999999, 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.123 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537072+0000", 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.124 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537073+0000", 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2750000000000001e-06 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537091+0000", 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.125 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537118+0000", 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.7220999999999999e-05 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537130+0000", 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1551e-05 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.126 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.537215+0000", 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.5156999999999998e-05 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.540378+0000", 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0031629000000000002 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.es0 33/32 ECSubRead(tid=7, to_read={3:7de15fb4:::smithi03327189-49:head=0,262144,0}, subchunks={3:7de15fb4:::smithi03327189-49:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:31.127 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.504639724, 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033061729999999999, 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545306+0000", 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545307+0000", 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.0699999999999999e-06 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545320+0000", 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.2867000000000001e-05 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.129 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545339+0000", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.9352e-05 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8869999999999998e-06 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.545406+0000", 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.7782999999999999e-05 2024-09-15T01:44:31.130 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.548612+0000", 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032062140000000002 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.7s1 33/32 ECSubRead(tid=46, to_read={3:e02ccbe4:::smithi03327189-36:head=0,262144,0}, subchunks={3:e02ccbe4:::smithi03327189-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.498773273000001, 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0033818339999999998, 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.131 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.132 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551172+0000", 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551173+0000", 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.111e-06 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551185+0000", 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.205e-05 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.133 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551202+0000", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6971999999999999e-05 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551209+0000", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5339999999999996e-06 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.551283+0000", 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.4664000000000002e-05 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.134 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.554554+0000", 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0032705030000000001 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=67, to_read={3:ac2d3331:::smithi03327189-18:head=0,262144,0}, subchunks={3:ac2d3331:::smithi03327189-18:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.496821529000002, 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003506547, 2024-09-15T01:44:31.135 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553124+0000", 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.136 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553125+0000", 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.2900000000000002e-07 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553136+0000", 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.1032999999999999e-05 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.137 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553161+0000", 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 2.5043000000000001e-05 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553170+0000", 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 8.8729999999999996e-06 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.138 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553235+0000", 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.5230999999999999e-05 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556630+0000", 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003395538 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.139 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.cs7 33/32 ECSubRead(tid=68, to_read={3:3584cd82:::smithi03327189-23:head=0,262144,0}, subchunks={3:3584cd82:::smithi03327189-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.496758081999999, 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0036502639999999999, 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:31.140 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553187+0000", 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553188+0000", 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 4.2599999999999998e-07 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.141 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553194+0000", 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 6.3770000000000001e-06 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553204+0000", 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.6819999999999998e-06 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.142 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553210+0000", 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.8850000000000004e-06 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.553352+0000", 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00014202600000000001 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.556838+0000", 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.003485868 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.143 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "description": "MOSDECSubOpRead(3.5s6 33/32 ECSubRead(tid=71, to_read={3:a22eae17:::smithi03327189-26:head=0,262144,0}, subchunks={3:a22eae17:::smithi03327189-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "initiated_at": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "age": 18.484454992, 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.0030758130000000002, 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "type_data": { 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "events": [ 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "initiated", 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:31.144 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "header_read", 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565490+0000", 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "throttled", 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565491+0000", 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 9.2699999999999998e-07 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "all_read", 2024-09-15T01:44:31.145 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565507+0000", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.6076e-05 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "dispatched", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565523+0000", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 1.5174000000000001e-05 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565529+0000", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 5.9819999999999996e-06 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "reached_pg", 2024-09-15T01:44:31.146 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.565605+0000", 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 7.6344e-05 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: { 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: "event": "done", 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: "time": "2024-09-15T01:44:12.568566+0000", 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: "duration": 0.00296131 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-09-15T01:44:31.147 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:31.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:31.149 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:31.144+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:31.250 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:31.244+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:31.259 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:31.259 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:31.259 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:31.260 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:31.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:31.351 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:31.343+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:31.377 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:31.377 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:31.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:31.452 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:31.448+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:31.488 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:31.488 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:31.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:31.488 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:31.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.489 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:31.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:31.489 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.955229053, 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:31.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:31.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:31.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:31.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.952797655000001, 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:31.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.952558840000002, 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:31.522 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.952527456999999, 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:31.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:31.529 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.973320027, 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:31.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.965405624999999, 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:31.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:31.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:31.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:31.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.965376362000001, 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:31.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:31.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:31.548 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.937939516, 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.936772771000001, 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.936399498, 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.935680399999999, 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.953465830000001, 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:31.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:31.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9658010430000008, 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:31.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:31.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9635625819999998, 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:31.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:31.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9629929130000008, 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:31.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:31.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:31.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:31.590 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9629339449999996, 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=13, reqid=client.4632.0:302, at_version=43'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:31.596 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9613410669999993, 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094416429999999996, 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520336+0000", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520377+0000", 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1192000000000001e-05 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520398+0000", 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1013e-05 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520411+0000", 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.239e-05 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522362+0000", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001951803 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522402+0000", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9764000000000001e-05 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529721+0000", 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007318504 2024-09-15T01:44:31.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529777+0000", 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6501e-05 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:31.601 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9475717780000004, 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083306100000000004, 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534128+0000", 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.315e-05 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534140+0000", 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1508999999999999e-05 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534146+0000", 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3230000000000003e-06 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534199+0000", 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3056999999999997e-05 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534209+0000", 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7999999999999993e-06 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542395+0000", 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081854519999999993 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542435+0000", 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0759000000000001e-05 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9750357369999998, 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:31.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9749419589999997, 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:31.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:31.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:31.619 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:31.620 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:31.511+0000 7f84b3625700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:44:31.621 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:31.511+0000 7f84b3625700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:44:31.621 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:31.515+0000 7f84af61d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:44:31.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:31.622 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:44:31.623 INFO:tasks.rados.rados.0.smithi033.stdout:170: done (1 left) 2024-09-15T01:44:31.623 INFO:tasks.rados.rados.0.smithi033.stdout:171: copy_from oid 15 from oid 29 current snap is 11 2024-09-15T01:44:31.623 INFO:tasks.rados.rados.0.smithi033.stdout:172: rollback oid 30 current snap is 11 2024-09-15T01:44:31.623 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 30 to 9 2024-09-15T01:44:31.623 INFO:tasks.rados.rados.0.smithi033.stdout:173: snap_remove snap 10 2024-09-15T01:44:31.624 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:31.612+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:31.722 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:31.715+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:31.769 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:31.769 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:31.780 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:31.822 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:31.815+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:31.884 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:31.884 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:31.884 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:31.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:31.923 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:31.915+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:31.999 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.473333748999998, 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:32.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.473294744, 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:32.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:32.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.473263800000002, 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=63, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:32.015 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.459784989999999, 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058134440000000001, 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534535+0000", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.13e-07 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534542+0000", 2024-09-15T01:44:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3890000000000002e-06 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534557+0000", 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4919000000000001e-05 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534562+0000", 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7189999999999998e-06 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540154+0000", 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055920869999999999 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540348+0000", 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019381699999999999 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.5s5 33/32 ECSubRead(tid=64, to_read={3:a51da153:::smithi03327189-47:head=0,196608,0}, subchunks={3:a51da153:::smithi03327189-47:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:32.019 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.459751703999999, 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059416290000000004, 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534568+0000", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8000000000000002e-07 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534573+0000", 2024-09-15T01:44:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2050000000000004e-06 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534579+0000", 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6479999999999997e-06 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.534584+0000", 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6079999999999998e-06 2024-09-15T01:44:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540397+0000", 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058133880000000001 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540510+0000", 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001125 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.458872680999999, 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:32.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.478379500000001, 2024-09-15T01:44:32.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:32.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=59, reqid=osd.3.0:59, at_version=38'16, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.464934723000001, 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062207900000000003, 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529385+0000", 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529386+0000", 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1200000000000002e-07 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529410+0000", 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4343000000000001e-05 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529422+0000", 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2128e-05 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529429+0000", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7750000000000004e-06 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529473+0000", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3523000000000003e-05 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.529479+0000", 2024-09-15T01:44:32.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3720000000000001e-06 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535566+0000", 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006087528 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.535606+0000", 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9409000000000002e-05 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/34 ECSubWrite(tid=60, reqid=osd.3.0:60, at_version=38'18, trim_to=0'0, roll_forward_to=37'14))", 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.462158311, 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060140539999999996, 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:32.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532162+0000", 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8100000000000001e-07 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532195+0000", 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2177e-05 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532208+0000", 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2991e-05 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532217+0000", 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8309999999999996e-06 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532269+0000", 2024-09-15T01:44:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080000000000003e-05 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532279+0000", 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6260000000000007e-06 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538153+0000", 2024-09-15T01:44:32.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058734499999999997 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.538176+0000", 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3017999999999999e-05 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.476383979, 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:32.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.476314447, 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:32.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.476291911000001, 2024-09-15T01:44:32.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:32.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.465997116, 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071464149999999997, 2024-09-15T01:44:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.06e-07 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528345+0000", 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1920999999999999e-05 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528355+0000", 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4830000000000005e-06 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528360+0000", 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5029999999999997e-06 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528430+0000", 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9640000000000004e-05 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528440+0000", 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0611999999999999e-05 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535446+0000", 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007005173 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535469+0000", 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3577000000000001e-05 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4762247189999993, 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:32.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:32.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4761551940000004, 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4759131159999992, 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4758596819999994, 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4757875029999994, 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4618259449999993, 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.4s0 43/34 ECSubReadReply(tid=50, attrs_read=0))", 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4602794360000004, 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063210219999999999, 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534040+0000", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534041+0000", 2024-09-15T01:44:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.37e-07 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537228+0000", 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031869820000000001 2024-09-15T01:44:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537240+0000", 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2371999999999999e-05 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537247+0000", 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8820000000000003e-06 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.539382+0000", 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021353000000000001 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540361+0000", 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097894900000000001 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.122 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:32.125 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:32.115+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:32.225 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:32.220+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:32.233 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:32.326 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:32.319+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:32.347 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.347 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:32.347 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:32.347 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:32.347 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:32.426 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:32.420+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.92038075, 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.461 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:32.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.465 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.939961673999999, 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040598090000000002, 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515911+0000", 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515912+0000", 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0800000000000005e-07 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515965+0000", 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3362999999999997e-05 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515984+0000", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9130000000000001e-05 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516042+0000", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7518000000000001e-05 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516184+0000", 2024-09-15T01:44:32.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014263599999999999 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516230+0000", 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5240000000000001e-05 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519914+0000", 2024-09-15T01:44:32.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036847939999999999 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.519971+0000", 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6619999999999997e-05 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:32.470 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.93661578, 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.936464852, 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.933862414, 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050197059999999996, 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522012+0000", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.122e-06 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522045+0000", 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3034999999999998e-05 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522067+0000", 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2742e-05 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522078+0000", 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0278000000000001e-05 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524135+0000", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020572450000000001 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524153+0000", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8510000000000001e-05 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527017+0000", 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028639719999999998 2024-09-15T01:44:32.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527030+0000", 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2802e-05 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=97, reqid=osd.5.0:95, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.927148218999999, 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041007480000000004, 2024-09-15T01:44:32.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528725+0000", 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528726+0000", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8700000000000004e-07 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528739+0000", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3694e-05 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528757+0000", 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501000000000001e-05 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528763+0000", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7270000000000001e-06 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528985+0000", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000221955 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.528996+0000", 2024-09-15T01:44:32.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0363000000000001e-05 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532796+0000", 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003800257 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.532825+0000", 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9363999999999999e-05 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.934130256, 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:32.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:32.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=34, reqid=client.4632.0:252, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.931606961, 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056439819999999996, 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524309+0000", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2670000000000003e-05 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524323+0000", 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3813e-05 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524331+0000", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2300000000000008e-06 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524482+0000", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015112700000000001 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524497+0000", 2024-09-15T01:44:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4666e-05 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529866+0000", 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053691579999999997 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529910+0000", 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3990999999999999e-05 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=35, reqid=client.4632.0:253, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.929679762999999, 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038225540000000001, 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526193+0000", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526194+0000", 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9099999999999991e-07 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526216+0000", 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2107e-05 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526228+0000", 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.169e-05 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526235+0000", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7309999999999996e-06 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526305+0000", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0108e-05 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526316+0000", 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1347e-05 2024-09-15T01:44:32.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529991+0000", 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036753570000000002 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530016+0000", 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4223000000000002e-05 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.7s1 43/34 ECSubRead(tid=73, to_read={3:e5da12c9:::smithi03327189-25:head=0,102400,0}, subchunks={3:e5da12c9:::smithi03327189-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9268281219999999, 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035108980000000001, 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529045+0000", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6800000000000001e-07 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529051+0000", 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1720000000000004e-06 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529064+0000", 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2435999999999999e-05 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529069+0000", 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2009999999999998e-06 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532343+0000", 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032738820000000001 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532556+0000", 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021283900000000001 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s4 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9230288340000001, 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046971579999999999, 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532845+0000", 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5600000000000005e-07 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532856+0000", 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1635e-05 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532868+0000", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1712000000000001e-05 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532874+0000", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3579999999999999e-06 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533006+0000", 2024-09-15T01:44:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013125800000000001 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533016+0000", 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537501+0000", 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044853799999999997 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537541+0000", 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790999999999999e-05 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9225479449999998, 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042471389999999996, 2024-09-15T01:44:32.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533325+0000", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9499999999999998e-07 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533350+0000", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.427e-05 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533361+0000", 2024-09-15T01:44:32.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1258e-05 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533367+0000", 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.438e-06 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533417+0000", 2024-09-15T01:44:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9325e-05 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533427+0000", 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0468999999999999e-05 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537550+0000", 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041229780000000002 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537572+0000", 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2106000000000001e-05 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=135, reqid=client.4632.0:304, at_version=43'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9068387579999992, 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059241329999999998, 2024-09-15T01:44:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549410+0000", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000375464 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549431+0000", 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713999999999999e-05 2024-09-15T01:44:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549438+0000", 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549499+0000", 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1303999999999994e-05 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549514+0000", 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4901e-05 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554929+0000", 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054145510000000001 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554958+0000", 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9393000000000001e-05 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.523 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/34 ECSubWrite(tid=54, reqid=client.4632.0:294, at_version=0'0, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9043917609999994, 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032223550000000001, 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551481+0000", 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551482+0000", 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551498+0000", 2024-09-15T01:44:32.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6297e-05 2024-09-15T01:44:32.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551514+0000", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5673999999999999e-05 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551522+0000", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6969999999999993e-06 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551573+0000", 2024-09-15T01:44:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1700999999999998e-05 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551584+0000", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554677+0000", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030922670000000001 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554703+0000", 2024-09-15T01:44:32.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6950000000000001e-05 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=136, reqid=client.4632.0:305, at_version=43'13, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9023621399999993, 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043787469999999997, 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553519+0000", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2460000000000003e-06 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554227+0000", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00070824899999999999 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554241+0000", 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4096e-05 2024-09-15T01:44:32.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554249+0000", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554324+0000", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5050999999999997e-05 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554335+0000", 2024-09-15T01:44:32.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0721e-05 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557866+0000", 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003531621 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557890+0000", 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3088999999999999e-05 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=137, reqid=client.4632.0:306, at_version=43'14, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007355090000004, 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046968319999999997, 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.536 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555138+0000", 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8599999999999998e-07 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555169+0000", 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0957000000000002e-05 2024-09-15T01:44:32.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555181+0000", 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1789e-05 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555188+0000", 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7479999999999996e-06 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555224+0000", 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5938e-05 2024-09-15T01:44:32.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555230+0000", 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.998e-06 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559806+0000", 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045767990000000003 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559834+0000", 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7917000000000001e-05 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=138, reqid=client.4632.0:307, at_version=43'15, trim_to=0'0, roll_forward_to=43'12))", 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8992040209999992, 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031781679999999999, 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556669+0000", 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556670+0000", 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3499999999999995e-07 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556689+0000", 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8981999999999999e-05 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556706+0000", 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7272999999999999e-05 2024-09-15T01:44:32.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556713+0000", 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8279999999999997e-06 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556755+0000", 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2341e-05 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.556764+0000", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8300000000000002e-06 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559837+0000", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030732849999999998 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559847+0000", 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 48/34 ECSubWrite(tid=157, reqid=client.4632.0:396, at_version=48'57, trim_to=0'0, roll_forward_to=46'55))", 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:27.519944+0000", 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9359286239999998, 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041757340000000004, 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.519944+0000", 2024-09-15T01:44:32.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.519944+0000", 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.519945+0000", 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3699999999999999e-07 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.520490+0000", 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054451300000000005 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.520529+0000", 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9657000000000003e-05 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.520548+0000", 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8906000000000002e-05 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.520715+0000", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016659699999999999 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.520740+0000", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4987999999999999e-05 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.524090+0000", 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033502110000000001 2024-09-15T01:44:32.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:27.524120+0000", 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9924999999999999e-05 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "pg_update_log_missing(3.cs5 epoch 52/51 rep_tid 93 entries 52'35 (0'0) error 3:37b49e87:::smithi03327189-48:head by client.4632.0:424 0.000000 -2 [r=-2+0b,r=0+0b] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.536287+0000", 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.91958625299999996, 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038708520000000001, 2024-09-15T01:44:32.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536287+0000", 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536287+0000", 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536287+0000", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0800000000000001e-07 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536293+0000", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0070000000000003e-06 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536307+0000", 2024-09-15T01:44:32.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4483e-05 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536320+0000", 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2503e-05 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.536393+0000", 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3036999999999999e-05 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.540158+0000", 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037646139999999999 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.8.17:1 3.es0 3:720ccbad:::smithi03327189-29:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.89629458900000003, 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047997630000000003, 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559580+0000", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.939e-06 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559588+0000", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4989999999999997e-06 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559605+0000", 2024-09-15T01:44:32.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559614+0000", 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6410000000000008e-06 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559684+0000", 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0072000000000003e-05 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559886+0000", 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020191600000000001 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.564378+0000", 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044920860000000002 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.562 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.564 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:32.556+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:32.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:32.565 INFO:tasks.rados.rados.0.smithi033.stdout:173: done (3 left) 2024-09-15T01:44:32.565 INFO:tasks.rados.rados.0.smithi033.stdout:174: delete oid 4 current snap is 11 2024-09-15T01:44:32.565 INFO:tasks.rados.rados.0.smithi033.stdout:172: finishing rollback tid 1 to smithi03327189-30 2024-09-15T01:44:32.565 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 30 v 0 (ObjNum 129 snap 0 seq_num 1161673256) dirty dne 2024-09-15T01:44:32.565 INFO:tasks.rados.rados.0.smithi033.stdout:169: expect (ObjNum 51 snap 0 seq_num 51) 2024-09-15T01:44:32.589 INFO:tasks.rados.rados.0.smithi033.stdout:171: finishing copy_from racing read to smithi03327189-15 2024-09-15T01:44:32.589 INFO:tasks.rados.rados.0.smithi033.stdout:171: finishing copy_from to smithi03327189-15 2024-09-15T01:44:32.589 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T01:44:32.660 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:32.661 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.665 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:32.660+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:32.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:32.766 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:32.763+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:32.815 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:32.816 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:32.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:32.867 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:32.860+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:32.920 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.407750918000001, 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.407696679000001, 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.930 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.407677540999998, 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:206 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~606208] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.397358729, 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062804979999999998, 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 206 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:32.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519793+0000", 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0190000000000001e-06 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520167+0000", 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037447000000000001 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520180+0000", 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3101e-05 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520186+0000", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9989999999999997e-06 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521198+0000", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001011807 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521308+0000", 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110059 2024-09-15T01:44:32.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522439+0000", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011307229999999999 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525693+0000", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003254159 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526006+0000", 2024-09-15T01:44:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000312971 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526071+0000", 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5190000000000004e-05 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:207 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.944 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.395655797, 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055629219999999997, 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 207 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521493+0000", 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521495+0000", 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.618e-06 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521721+0000", 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022585199999999999 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521741+0000", 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0496000000000002e-05 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521749+0000", 2024-09-15T01:44:32.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2620000000000001e-06 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523331+0000", 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015822919999999999 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.523435+0000", 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001044 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524251+0000", 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000815367 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526200+0000", 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019496120000000001 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526995+0000", 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079458399999999998 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527056+0000", 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1439000000000006e-05 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:208 3.2s0 3:47035a48:::smithi03327189-20:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.395389955999999, 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056281209999999998, 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 208 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521759+0000", 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521760+0000", 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8700000000000005e-07 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521765+0000", 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521774+0000", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8240000000000006e-06 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521778+0000", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3760000000000001e-06 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.524887+0000", 2024-09-15T01:44:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031084820000000001 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525036+0000", 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000149736 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525431+0000", 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039431399999999999 2024-09-15T01:44:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527107+0000", 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001676147 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527310+0000", 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000203632 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527387+0000", 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6938000000000001e-05 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:209 3.2s0 3:47035a48:::smithi03327189-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.395367078, 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079111819999999992, 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 209 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:32.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521783+0000", 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521784+0000", 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.736e-06 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521791+0000", 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0849999999999999e-06 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521803+0000", 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526224+0000", 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004421431 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526350+0000", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012618500000000001 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527766+0000", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014151299999999999 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527853+0000", 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7435999999999994e-05 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529693+0000", 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001840436 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0, backfill_or_async_recovery))", 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.384974528000001, 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083125319999999992, 2024-09-15T01:44:32.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:32.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532176+0000", 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2191000000000001e-05 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6229999999999999e-06 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532292+0000", 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3135999999999999e-05 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532304+0000", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2481e-05 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540422+0000", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081175480000000005 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540487+0000", 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5278999999999996e-05 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.38187542, 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077245059999999999, 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535275+0000", 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000001e-07 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535296+0000", 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1590000000000002e-05 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535309+0000", 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2247e-05 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535315+0000", 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7870000000000004e-06 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535399+0000", 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3398000000000003e-05 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535410+0000", 2024-09-15T01:44:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542978+0000", 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075673989999999998 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542998+0000", 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0469999999999999e-05 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=2, reqid=client.4632.0:220, at_version=33'6, trim_to=0'0, roll_forward_to=33'4, backfill_or_async_recovery))", 2024-09-15T01:44:32.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.380310983000001, 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061654960000000003, 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536838+0000", 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536839+0000", 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6899999999999998e-07 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536859+0000", 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0585999999999999e-05 2024-09-15T01:44:32.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536869+0000", 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3989999999999993e-06 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536878+0000", 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6180000000000001e-06 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536929+0000", 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0285999999999999e-05 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.536939+0000", 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0472000000000001e-05 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542974+0000", 2024-09-15T01:44:32.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006034569 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543004+0000", 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0196999999999999e-05 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.396498243, 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065983789999999997, 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520652+0000", 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0330000000000001e-06 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520685+0000", 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3210999999999998e-05 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520702+0000", 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6880000000000001e-05 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520712+0000", 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0003000000000001e-05 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520819+0000", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106947 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520835+0000", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5647000000000001e-05 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527222+0000", 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063877339999999999 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527249+0000", 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6923999999999998e-05 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 38/34 ECSubWrite(tid=42, reqid=osd.0.0:42, at_version=38'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.392535855, 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060356519999999999, 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524613+0000", 2024-09-15T01:44:32.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524614+0000", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7999999999999996e-07 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524667+0000", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2803000000000002e-05 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524683+0000", 2024-09-15T01:44:32.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6623e-05 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524692+0000", 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5169999999999999e-06 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524799+0000", 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010671300000000001 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524810+0000", 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1661999999999999e-05 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530622+0000", 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058122499999999997 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.530649+0000", 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6604000000000001e-05 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.398702641, 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:32.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.398634979000001, 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:32.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:33.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.398609238000001, 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:33.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.385538429, 2024-09-15T01:44:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:33.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:33.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.384496523999999, 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.383675718999999, 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:33.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.5s1 52/51 ECSubReadReply(tid=96, attrs_read=0))", 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3737002490000001, 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077934340000000001, 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543464+0000", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4994000000000001e-05 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543468+0000", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.002e-06 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.547336+0000", 2024-09-15T01:44:33.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003867919 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.551242+0000", 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039065189999999998 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:427 3.cs1 3:3624ba55:::smithi03327189-30:head [rollback 13] snapc 15=[15,13] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.543967+0000", 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.373182363, 2024-09-15T01:44:33.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062498600000000003, 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 427 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543967+0000", 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543967+0000", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543968+0000", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3960000000000001e-06 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543973+0000", 2024-09-15T01:44:33.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4150000000000001e-06 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543983+0000", 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0131e-05 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543988+0000", 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2839999999999997e-06 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.544030+0000", 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2219999999999999e-05 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for missing object", 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.544078+0000", 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7240000000000002e-05 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.547104+0000", 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030260769999999998 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.547300+0000", 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000196529 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.550217+0000", 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002916568 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:33.032 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:33.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:33.034 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:33.031+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:33.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:33.134 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:33.128+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:429 3.0s0 3.e100d650 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:32.539474+0000", 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.60457106400000005, 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.60462341900000005, 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "waiting for peered", 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 429 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539474+0000", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539474+0000", 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539478+0000", 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1970000000000001e-06 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539498+0000", 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0829e-05 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539565+0000", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6612000000000005e-05 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539585+0000", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0336e-05 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539696+0000", 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000111097 2024-09-15T01:44:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:32.539701+0000", 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.9830000000000004e-06 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-09-15T01:44:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 1 2024-09-15T01:44:33.151 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:33.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:33.216 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:33.235 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:33.227+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:33.236 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:33.253 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:33.253 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:33.255 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:33.262 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:33.272 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:33.291 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:33.311 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:33.331 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:33.337 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:33.331+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:33.354 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:33.366 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.366 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:33.377 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:33.437 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:33.435+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:33.478 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.478 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:33.538 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:33.532+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:33.561 INFO:tasks.rados.rados.0.smithi033.stdout:169: done (3 left) 2024-09-15T01:44:33.561 INFO:tasks.rados.rados.0.smithi033.stdout:171: done (2 left) 2024-09-15T01:44:33.561 INFO:tasks.rados.rados.0.smithi033.stdout:172: done (1 left) 2024-09-15T01:44:33.561 INFO:tasks.rados.rados.0.smithi033.stdout:174: done (0 left) 2024-09-15T01:44:33.561 INFO:tasks.rados.rados.0.smithi033.stdout:175: rmattr oid 25 current snap is 11 2024-09-15T01:44:33.563 INFO:tasks.rados.rados.0.smithi033.stdout:176: setattr oid 16 current snap is 11 2024-09-15T01:44:33.563 INFO:tasks.rados.rados.0.smithi033.stdout:177: snap_remove snap 9 2024-09-15T01:44:33.601 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:33.605 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.605 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:33.621 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:33.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:44:33.638 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:33.631+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:33.648 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:33.650 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:33.651 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:33.651 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:33.651 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:33.651 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:33.651 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:33.710 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.711 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:33.740 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:33.731+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:33.814 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.814 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:33.840 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:33.835+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:33.922 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:33.922 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:33.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:33.941 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:33.936+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:34.035 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.035 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.042 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:34.036+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:34.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:34.143 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:34.136+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:34.187 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.187 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.197 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:34.243 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:34.236+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:34.297 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.298 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:34.349 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:34.344+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:34.446 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:34.443+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:34.451 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.452 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:34.546 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:34.539+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:34.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:34.547 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:34.547 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-15T01:44:34.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:34.555 INFO:tasks.rados.rados.0.smithi033.stdout:177: done (2 left) 2024-09-15T01:44:34.555 INFO:tasks.rados.rados.0.smithi033.stdout:178: read oid 40 snap -1 2024-09-15T01:44:34.555 INFO:tasks.rados.rados.0.smithi033.stdout:178: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-15T01:44:34.555 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 22 current snap is 11 2024-09-15T01:44:34.555 INFO:tasks.rados.rados.0.smithi033.stdout:179: seq_num 92 ranges {0=786432,786432=491520,1277952=278528} 2024-09-15T01:44:34.563 INFO:tasks.rados.rados.0.smithi033.stdout:179: writing smithi03327189-22 from 0 to 786432 tid 1 2024-09-15T01:44:34.568 INFO:tasks.rados.rados.0.smithi033.stdout:179: writing smithi03327189-22 from 786432 to 1277952 tid 2 2024-09-15T01:44:34.572 INFO:tasks.rados.rados.0.smithi033.stdout:179: writing smithi03327189-22 from 1277952 to 1556480 tid 3 2024-09-15T01:44:34.572 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 16 v 8 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 25 v 18 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:175: done (3 left) 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:176: done (2 left) 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:180: rmattr oid 18 current snap is 11 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:180: done (2 left) 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 2 current snap is 11 2024-09-15T01:44:34.573 INFO:tasks.rados.rados.0.smithi033.stdout:181: seq_num 93 ranges {32768=802816,835584=638976,1474560=131072} 2024-09-15T01:44:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:44:34.585 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.585 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.589 INFO:tasks.rados.rados.0.smithi033.stdout:181: writing smithi03327189-2 from 32768 to 835584 tid 1 2024-09-15T01:44:34.592 INFO:tasks.rados.rados.0.smithi033.stdout:181: writing smithi03327189-2 from 835584 to 1474560 tid 2 2024-09-15T01:44:34.597 INFO:tasks.rados.rados.0.smithi033.stdout:181: writing smithi03327189-2 from 1474560 to 1605632 tid 3 2024-09-15T01:44:34.597 INFO:tasks.rados.rados.0.smithi033.stdout:182: read oid 20 snap -1 2024-09-15T01:44:34.597 INFO:tasks.rados.rados.0.smithi033.stdout:182: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-15T01:44:34.597 INFO:tasks.rados.rados.0.smithi033.stdout:183: copy_from oid 9 from oid 4 current snap is 11 2024-09-15T01:44:34.597 INFO:tasks.rados.rados.0.smithi033.stdout:178: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-15T01:44:34.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: finishing write tid 1 to smithi03327189-22 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: finishing write tid 2 to smithi03327189-22 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: finishing write tid 3 to smithi03327189-22 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: finishing write tid 4 to smithi03327189-22 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: finishing write tid 5 to smithi03327189-22 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 22 v 25 (ObjNum 92 snap 11 seq_num 92) dirty exists 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:179: left oid 22 (ObjNum 92 snap 11 seq_num 92) 2024-09-15T01:44:34.618 INFO:tasks.rados.rados.0.smithi033.stdout:183: finishing copy_from to smithi03327189-9 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:183: got expected ENOENT (src dne) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:183: finishing copy_from racing read to smithi03327189-9 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:181: finishing write tid 1 to smithi03327189-2 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:178: done (4 left) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:179: done (3 left) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:183: done (2 left) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:184: rmattr oid 40 current snap is 11 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:184: done (2 left) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:185: rollback oid 35 current snap is 11 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:185: done (2 left) 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 14 current snap is 11 2024-09-15T01:44:34.619 INFO:tasks.rados.rados.0.smithi033.stdout:182: expect (ObjNum 58 snap 2 seq_num 58) 2024-09-15T01:44:34.644 INFO:tasks.rados.rados.0.smithi033.stdout:181: finishing write tid 2 to smithi03327189-2 2024-09-15T01:44:34.644 INFO:tasks.rados.rados.0.smithi033.stdout:181: finishing write tid 3 to smithi03327189-2 2024-09-15T01:44:34.644 INFO:tasks.rados.rados.0.smithi033.stdout:181: finishing write tid 4 to smithi03327189-2 2024-09-15T01:44:34.645 INFO:tasks.rados.rados.0.smithi033.stdout:181: finishing write tid 5 to smithi03327189-2 2024-09-15T01:44:34.645 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 2 v 28 (ObjNum 93 snap 11 seq_num 93) dirty exists 2024-09-15T01:44:34.645 INFO:tasks.rados.rados.0.smithi033.stdout:181: left oid 2 (ObjNum 93 snap 11 seq_num 93) 2024-09-15T01:44:34.645 INFO:tasks.rados.rados.0.smithi033.stdout:186: seq_num 94 ranges {2490368=425984,2916352=786432,3702784=376832} 2024-09-15T01:44:34.647 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:34.640+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:34.668 INFO:tasks.rados.rados.0.smithi033.stdout:186: writing smithi03327189-14 from 2490368 to 2916352 tid 1 2024-09-15T01:44:34.673 INFO:tasks.rados.rados.0.smithi033.stdout:186: writing smithi03327189-14 from 2916352 to 3702784 tid 2 2024-09-15T01:44:34.676 INFO:tasks.rados.rados.0.smithi033.stdout:186: writing smithi03327189-14 from 3702784 to 4079616 tid 3 2024-09-15T01:44:34.676 INFO:tasks.rados.rados.0.smithi033.stdout:181: done (2 left) 2024-09-15T01:44:34.676 INFO:tasks.rados.rados.0.smithi033.stdout:182: done (1 left) 2024-09-15T01:44:34.676 INFO:tasks.rados.rados.0.smithi033.stdout:187: snap_create 2024-09-15T01:44:34.676 INFO:tasks.rados.rados.0.smithi033.stdout:186: finishing write tid 1 to smithi03327189-14 2024-09-15T01:44:34.680 INFO:tasks.rados.rados.0.smithi033.stdout:186: finishing write tid 2 to smithi03327189-14 2024-09-15T01:44:34.682 INFO:tasks.rados.rados.0.smithi033.stdout:186: finishing write tid 3 to smithi03327189-14 2024-09-15T01:44:34.682 INFO:tasks.rados.rados.0.smithi033.stdout:186: finishing write tid 4 to smithi03327189-14 2024-09-15T01:44:34.685 INFO:tasks.rados.rados.0.smithi033.stdout:186: finishing write tid 5 to smithi03327189-14 2024-09-15T01:44:34.685 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 38 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:44:34.686 INFO:tasks.rados.rados.0.smithi033.stdout:186: left oid 14 (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:44:34.693 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.693 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.702 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:34.748 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:34.744+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:34.794 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.795 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.803 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:34.850 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:34.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:34.844+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:34.852 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:34.868 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-15T01:44:34.548065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":46121523,"num_objects":49,"num_object_clones":4,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":1,"num_read":272,"num_read_kb":28405,"num_write":418,"num_write_kb":57016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19583520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":146,"acting":145,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":10,"kb":943718400,"kb_used":3060916,"kb_used_data":87284,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":940657484,"statfs":{"total":966367641600,"available":963233263616,"internally_reserved":0,"allocated":89378816,"data_stored":91971685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5750784,"num_objects":5,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":8,"num_read_kb":99,"num_write":17,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035473"},"pg_stats":[{"pgid":"3.f","version":"46'24","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T01:44:32.542856+0000","last_change":"2024-09-15T01:44:32.543374+0000","last_active":"2024-09-15T01:44:32.542856+0000","last_peered":"2024-09-15T01:44:32.542856+0000","last_clean":"2024-09-15T01:44:32.542856+0000","last_became_active":"2024-09-15T01:44:14.525529+0000","last_became_peered":"2024-09-15T01:44:14.525529+0000","last_unstale":"2024-09-15T01:44:32.542856+0000","last_undegraded":"2024-09-15T01:44:32.542856+0000","last_fullsized":"2024-09-15T01:44:32.542856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067264199999999999,"stat_sum":{"num_bytes":4587520,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":2168,"num_write":18,"num_write_kb":5152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,1,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.e","version":"40'9","reported_seq":100,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.511877+0000","last_change":"2024-09-15T01:44:28.509364+0000","last_active":"2024-09-15T01:44:29.511877+0000","last_peered":"2024-09-15T01:44:29.511877+0000","last_clean":"2024-09-15T01:44:29.511877+0000","last_became_active":"2024-09-15T01:44:14.525437+0000","last_became_peered":"2024-09-15T01:44:14.525437+0000","last_unstale":"2024-09-15T01:44:29.511877+0000","last_undegraded":"2024-09-15T01:44:29.511877+0000","last_fullsized":"2024-09-15T01:44:29.511877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.758952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661310000000001,"stat_sum":{"num_bytes":475136,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":6,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.d","version":"30'6","reported_seq":150,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-15T01:44:32.541141+0000","last_change":"2024-09-15T01:44:32.541815+0000","last_active":"2024-09-15T01:44:32.541024+0000","last_peered":"2024-09-15T01:44:31.547656+0000","last_clean":"2024-09-15T01:44:28.534050+0000","last_became_active":"2024-09-15T01:44:31.522769+0000","last_became_peered":"2024-09-15T01:44:31.522769+0000","last_unstale":"2024-09-15T01:44:32.541141+0000","last_undegraded":"2024-09-15T01:44:32.541141+0000","last_fullsized":"2024-09-15T01:44:32.541141+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:07.871223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00204842,"stat_sum":{"num_bytes":1540096,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3080192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,9,6,1,3,4,0],"acting":[7,10,9,6,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,4,5,6,9,10],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"1"}]},{"pgid":"3.c","version":"46'34","reported_seq":146,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.526662+0000","last_change":"2024-09-15T01:44:30.527152+0000","last_active":"2024-09-15T01:44:30.526616+0000","last_peered":"2024-09-15T01:44:25.602340+0000","last_clean":"2024-09-15T01:44:25.602340+0000","last_became_active":"2024-09-15T01:44:14.524938+0000","last_became_peered":"2024-09-15T01:44:14.524938+0000","last_unstale":"2024-09-15T01:44:30.526662+0000","last_undegraded":"2024-09-15T01:44:30.526662+0000","last_fullsized":"2024-09-15T01:44:30.526662+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.257796+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015652329,"stat_sum":{"num_bytes":4112384,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":6,"num_write":19,"num_write_kb":6432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,10,9,2,0,7],"acting":[2147483647,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,4,5,6,7,9,10],"up_primary":4,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.b","version":"40'25","reported_seq":138,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.520730+0000","last_change":"2024-09-15T01:44:28.532467+0000","last_active":"2024-09-15T01:44:29.520730+0000","last_peered":"2024-09-15T01:44:29.520730+0000","last_clean":"2024-09-15T01:44:29.520730+0000","last_became_active":"2024-09-15T01:44:14.534602+0000","last_became_peered":"2024-09-15T01:44:14.534602+0000","last_unstale":"2024-09-15T01:44:29.520730+0000","last_undegraded":"2024-09-15T01:44:29.520730+0000","last_fullsized":"2024-09-15T01:44:29.520730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:17.646194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018465350000000001,"stat_sum":{"num_bytes":2637824,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":30,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.a","version":"46'10","reported_seq":93,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.527984+0000","last_change":"2024-09-15T01:44:28.509745+0000","last_active":"2024-09-15T01:44:29.527984+0000","last_peered":"2024-09-15T01:44:29.527984+0000","last_clean":"2024-09-15T01:44:29.527984+0000","last_became_active":"2024-09-15T01:44:07.579889+0000","last_became_peered":"2024-09-15T01:44:07.579889+0000","last_unstale":"2024-09-15T01:44:29.527984+0000","last_undegraded":"2024-09-15T01:44:29.527984+0000","last_fullsized":"2024-09-15T01:44:29.527984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.248888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068771,"stat_sum":{"num_bytes":491520,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":7,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.9","version":"38'8","reported_seq":74,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.517421+0000","last_change":"2024-09-15T01:44:30.517945+0000","last_active":"2024-09-15T01:44:30.517302+0000","last_peered":"2024-09-15T01:44:17.533201+0000","last_clean":"2024-09-15T01:44:17.533201+0000","last_became_active":"2024-09-15T01:44:14.532160+0000","last_became_peered":"2024-09-15T01:44:14.532160+0000","last_unstale":"2024-09-15T01:44:30.517421+0000","last_undegraded":"2024-09-15T01:44:30.517421+0000","last_fullsized":"2024-09-15T01:44:30.517421+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:26.714216+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0026602470000000001,"stat_sum":{"num_bytes":1261568,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,0,1,4,10],"acting":[8,3,7,9,2147483647,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,4,5,7,9,10],"up_primary":3,"acting_primary":8,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.8","version":"46'18","reported_seq":127,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.517976+0000","last_change":"2024-09-15T01:44:30.518600+0000","last_active":"2024-09-15T01:44:30.517920+0000","last_peered":"2024-09-15T01:44:25.601723+0000","last_clean":"2024-09-15T01:44:25.601723+0000","last_became_active":"2024-09-15T01:44:14.537683+0000","last_became_peered":"2024-09-15T01:44:14.537683+0000","last_unstale":"2024-09-15T01:44:30.517976+0000","last_undegraded":"2024-09-15T01:44:30.517976+0000","last_fullsized":"2024-09-15T01:44:30.517976+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:11.692086+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.020140881999999999,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":2547,"num_write":8,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,9,10,7],"acting":[8,1,2,4,3,2147483647,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,4,5,7,9,10],"up_primary":1,"acting_primary":8,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":178,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:44:31.508605+0000","last_change":"2024-09-15T01:44:14.520601+0000","last_active":"2024-09-15T01:44:31.508605+0000","last_peered":"2024-09-15T01:44:31.508605+0000","last_clean":"2024-09-15T01:44:31.508605+0000","last_became_active":"2024-09-15T01:44:14.520194+0000","last_became_peered":"2024-09-15T01:44:14.520194+0000","last_unstale":"2024-09-15T01:44:31.508605+0000","last_undegraded":"2024-09-15T01:44:31.508605+0000","last_fullsized":"2024-09-15T01:44:31.508605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:59.848114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T01:44:30.535121+0000","last_change":"2024-09-15T01:44:30.535121+0000","last_active":"2024-09-15T01:44:30.535121+0000","last_peered":"2024-09-15T01:44:30.535121+0000","last_clean":"2024-09-15T01:44:30.535121+0000","last_became_active":"2024-09-15T01:44:30.530642+0000","last_became_peered":"2024-09-15T01:44:30.530642+0000","last_unstale":"2024-09-15T01:44:30.535121+0000","last_undegraded":"2024-09-15T01:44:30.535121+0000","last_fullsized":"2024-09-15T01:44:30.535121+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"46'20","reported_seq":103,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.525209+0000","last_change":"2024-09-15T01:44:30.525697+0000","last_active":"2024-09-15T01:44:29.522359+0000","last_peered":"2024-09-15T01:44:28.531323+0000","last_clean":"2024-09-15T01:44:28.531323+0000","last_became_active":"2024-09-15T01:44:07.579340+0000","last_became_peered":"2024-09-15T01:44:07.579340+0000","last_unstale":"2024-09-15T01:44:30.525209+0000","last_undegraded":"2024-09-15T01:44:30.525209+0000","last_fullsized":"2024-09-15T01:44:30.525209+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:18.421086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095284700000000005,"stat_sum":{"num_bytes":1523712,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1588,"num_write":12,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,8,1,2,10],"acting":[3,9,6,4,2147483647,1,2147483647,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4,5,6,8,9,10],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T01:44:31.507277+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:44:31.507277+0000","last_peered":"2024-09-15T01:44:31.507277+0000","last_clean":"2024-09-15T01:44:31.507277+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:44:31.507277+0000","last_undegraded":"2024-09-15T01:44:31.507277+0000","last_fullsized":"2024-09-15T01:44:31.507277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"49'32","reported_seq":164,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.519722+0000","last_change":"2024-09-15T01:44:30.520744+0000","last_active":"2024-09-15T01:44:29.528168+0000","last_peered":"2024-09-15T01:44:28.573740+0000","last_clean":"2024-09-15T01:44:28.573740+0000","last_became_active":"2024-09-15T01:44:14.529293+0000","last_became_peered":"2024-09-15T01:44:14.529293+0000","last_unstale":"2024-09-15T01:44:30.519722+0000","last_undegraded":"2024-09-15T01:44:30.519722+0000","last_fullsized":"2024-09-15T01:44:30.519722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:19.849442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013416389999999999,"stat_sum":{"num_bytes":5193728,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":3656,"num_write":20,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,8,9,6,1],"acting":[0,4,10,3,2147483647,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,4,5,6,8,9,10],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.511559+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:44:29.511559+0000","last_peered":"2024-09-15T01:44:29.511559+0000","last_clean":"2024-09-15T01:44:29.511559+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:44:29.511559+0000","last_undegraded":"2024-09-15T01:44:29.511559+0000","last_fullsized":"2024-09-15T01:44:29.511559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"46'23","reported_seq":119,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.522206+0000","last_change":"2024-09-15T01:44:30.522643+0000","last_active":"2024-09-15T01:44:30.522158+0000","last_peered":"2024-09-15T01:44:25.588866+0000","last_clean":"2024-09-15T01:44:25.588866+0000","last_became_active":"2024-09-15T01:44:14.525102+0000","last_became_peered":"2024-09-15T01:44:14.525102+0000","last_unstale":"2024-09-15T01:44:30.522206+0000","last_undegraded":"2024-09-15T01:44:30.522206+0000","last_fullsized":"2024-09-15T01:44:30.522206+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:31.661029+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01027105,"stat_sum":{"num_bytes":3211264,"num_objects":5,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":3705,"num_write":17,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,6,1,7,8,0,3],"acting":[2147483647,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,4,5,6,7,8],"up_primary":4,"acting_primary":9,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":18,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.525644+0000","last_change":"2024-09-15T01:44:30.525999+0000","last_active":"2024-09-15T01:44:29.517127+0000","last_peered":"2024-09-15T01:43:57.413501+0000","last_clean":"2024-09-15T01:43:57.413501+0000","last_became_active":"2024-09-15T01:43:55.396828+0000","last_became_peered":"2024-09-15T01:43:55.396828+0000","last_unstale":"2024-09-15T01:44:30.525644+0000","last_undegraded":"2024-09-15T01:44:30.525644+0000","last_fullsized":"2024-09-15T01:44:30.525644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"46'15","reported_seq":133,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.522050+0000","last_change":"2024-09-15T01:44:30.522604+0000","last_active":"2024-09-15T01:44:29.518867+0000","last_peered":"2024-09-15T01:44:28.553174+0000","last_clean":"2024-09-15T01:44:28.553174+0000","last_became_active":"2024-09-15T01:44:14.527181+0000","last_became_peered":"2024-09-15T01:44:14.527181+0000","last_unstale":"2024-09-15T01:44:30.522050+0000","last_undegraded":"2024-09-15T01:44:30.522050+0000","last_fullsized":"2024-09-15T01:44:30.522050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.647645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054945299999999999,"stat_sum":{"num_bytes":1622016,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1347,"num_write":9,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,8,9,10],"acting":[4,3,0,6,2,2147483647,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,3,5,6,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"3.5","version":"49'67","reported_seq":201,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.528122+0000","last_change":"2024-09-15T01:44:30.528561+0000","last_active":"2024-09-15T01:44:30.528067+0000","last_peered":"2024-09-15T01:44:28.570252+0000","last_clean":"2024-09-15T01:44:28.570252+0000","last_became_active":"2024-09-15T01:44:14.526182+0000","last_became_peered":"2024-09-15T01:44:14.526182+0000","last_unstale":"2024-09-15T01:44:30.528122+0000","last_undegraded":"2024-09-15T01:44:30.528122+0000","last_fullsized":"2024-09-15T01:44:30.528122+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:12.171051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018453863000000001,"stat_sum":{"num_bytes":8568832,"num_objects":10,"num_object_clones":2,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":26,"num_read_kb":2317,"num_write":43,"num_write_kb":10384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5160960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,6,7,1,8,10],"acting":[2147483647,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,5,6,7,8,9,10],"up_primary":9,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.518034+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:44:29.518034+0000","last_peered":"2024-09-15T01:44:29.518034+0000","last_clean":"2024-09-15T01:44:29.518034+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:44:29.518034+0000","last_undegraded":"2024-09-15T01:44:29.518034+0000","last_fullsized":"2024-09-15T01:44:29.518034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"49'39","reported_seq":167,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.525649+0000","last_change":"2024-09-15T01:44:30.526052+0000","last_active":"2024-09-15T01:44:29.517609+0000","last_peered":"2024-09-15T01:44:28.559969+0000","last_clean":"2024-09-15T01:44:28.559969+0000","last_became_active":"2024-09-15T01:44:14.526114+0000","last_became_peered":"2024-09-15T01:44:14.526114+0000","last_unstale":"2024-09-15T01:44:30.525649+0000","last_undegraded":"2024-09-15T01:44:30.525649+0000","last_fullsized":"2024-09-15T01:44:30.525649+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:19.309759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010977630000000001,"stat_sum":{"num_bytes":6176768,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":9448,"num_write":49,"num_write_kb":6304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,3,10,2,6,0],"acting":[1,8,9,2147483647,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,3,5,6,8,9,10],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.527015+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:44:29.527015+0000","last_peered":"2024-09-15T01:44:29.527015+0000","last_clean":"2024-09-15T01:44:29.527015+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:44:29.527015+0000","last_undegraded":"2024-09-15T01:44:29.527015+0000","last_fullsized":"2024-09-15T01:44:29.527015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"46'16","reported_seq":133,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.521632+0000","last_change":"2024-09-15T01:44:28.532795+0000","last_active":"2024-09-15T01:44:29.521632+0000","last_peered":"2024-09-15T01:44:29.521632+0000","last_clean":"2024-09-15T01:44:29.521632+0000","last_became_active":"2024-09-15T01:44:14.534165+0000","last_became_peered":"2024-09-15T01:44:14.534165+0000","last_unstale":"2024-09-15T01:44:29.521632+0000","last_undegraded":"2024-09-15T01:44:29.521632+0000","last_fullsized":"2024-09-15T01:44:29.521632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.075163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021497479999999999,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1255,"num_write":29,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.518694+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:44:29.518694+0000","last_peered":"2024-09-15T01:44:29.518694+0000","last_clean":"2024-09-15T01:44:29.518694+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:44:29.518694+0000","last_undegraded":"2024-09-15T01:44:29.518694+0000","last_fullsized":"2024-09-15T01:44:29.518694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"49'21","reported_seq":104,"reported_epoch":51,"state":"remapped+peering","last_fresh":"2024-09-15T01:44:30.521497+0000","last_change":"2024-09-15T01:44:30.522649+0000","last_active":"2024-09-15T01:44:29.526561+0000","last_peered":"2024-09-15T01:44:28.554866+0000","last_clean":"2024-09-15T01:44:28.554866+0000","last_became_active":"2024-09-15T01:44:07.578405+0000","last_became_peered":"2024-09-15T01:44:07.578405+0000","last_unstale":"2024-09-15T01:44:30.521497+0000","last_undegraded":"2024-09-15T01:44:30.521497+0000","last_fullsized":"2024-09-15T01:44:30.521497+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:11.856040+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006647086,"stat_sum":{"num_bytes":1671168,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":5,"num_write":15,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,6],"acting":[0,1,4,7,8,10,9,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,5,6,7,8,9,10],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"12","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.521760+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:44:29.521760+0000","last_peered":"2024-09-15T01:44:29.521760+0000","last_clean":"2024-09-15T01:44:29.521760+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:44:29.521760+0000","last_undegraded":"2024-09-15T01:44:29.521760+0000","last_fullsized":"2024-09-15T01:44:29.521760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":45531136,"num_objects":45,"num_object_clones":4,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":1,"num_read":196,"num_read_kb":28341,"num_write":293,"num_write_kb":54864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":19054592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96100352,"data_stored":102031360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":367,"ondisk_log_size":367,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":15,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306056,"kb_used_data":8692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065784,"statfs":{"total":96636764160,"available":96323362816,"internally_reserved":0,"allocated":8900608,"data_stored":8700432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444042,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444043,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307248,"kb_used_data":9884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064592,"statfs":{"total":96636764160,"available":96322142208,"internally_reserved":0,"allocated":10121216,"data_stored":10603422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]},{"osd":9,"up_from":17,"seq":73014444042,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306960,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064880,"statfs":{"total":96636764160,"available":96322437120,"internally_reserved":0,"allocated":9826304,"data_stored":10234782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305544,"kb_used_data":8180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066296,"statfs":{"total":96636764160,"available":96323887104,"internally_reserved":0,"allocated":8376320,"data_stored":8563614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444043,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307020,"kb_used_data":9656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064820,"statfs":{"total":96636764160,"available":96322375680,"internally_reserved":0,"allocated":9887744,"data_stored":10366398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8089999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.954}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.71}]}]},{"osd":5,"up_from":0,"seq":73014444043,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":17,"seq":73014444042,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303664,"kb_used_data":6300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068176,"statfs":{"total":96636764160,"available":96325812224,"internally_reserved":0,"allocated":6451200,"data_stored":6486942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0270000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444042,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305996,"kb_used_data":8636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065844,"statfs":{"total":96636764160,"available":96323424256,"internally_reserved":0,"allocated":8843264,"data_stored":8959810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.73}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8779999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444043,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304772,"kb_used_data":7408,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067068,"statfs":{"total":96636764160,"available":96324677632,"internally_reserved":0,"allocated":7585792,"data_stored":7773086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444042,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":10292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":10539008,"data_stored":11017137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9590000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306000,"kb_used_data":8640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065840,"statfs":{"total":96636764160,"available":96323420160,"internally_reserved":0,"allocated":8847360,"data_stored":9266062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8282112,"data_stored":8949760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9949184,"data_stored":10690560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7000064,"data_stored":7446528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8273920,"data_stored":8642560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5865472,"data_stored":6160384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8859648,"data_stored":9601024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8769536,"data_stored":9510912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7753728,"data_stored":7827456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":7790592,"data_stored":8237056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9240576,"data_stored":9908224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":9535488,"data_stored":10276864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:34.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-15T01:44:34.905 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:34.905 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:34.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:34.950 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:34.943+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:35.015 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.015 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.024 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:35.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:35.044+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:35.117 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.117 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:35.152 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:35.144+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:35.224 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.224 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.233 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:35.252 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:35.248+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:35.329 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.330 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:35.388 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:35.383+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:35.436 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.437 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.446 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:35.455 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:35.452+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:35.546 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.546 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.554 INFO:tasks.rados.rados.0.smithi033.stdout:186: done (1 left) 2024-09-15T01:44:35.554 INFO:tasks.rados.rados.0.smithi033.stdout:187: done (0 left) 2024-09-15T01:44:35.554 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 35 current snap is 12 2024-09-15T01:44:35.554 INFO:tasks.rados.rados.0.smithi033.stdout:188: seq_num 95 ranges {2490368=442368,2932736=540672,3473408=622592} 2024-09-15T01:44:35.555 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:35.547+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:35.556 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 3 pgp_num to 16 2024-09-15T01:44:35.580 INFO:tasks.rados.rados.0.smithi033.stdout:188: writing smithi03327189-35 from 2490368 to 2932736 tid 1 2024-09-15T01:44:35.583 INFO:tasks.rados.rados.0.smithi033.stdout:188: writing smithi03327189-35 from 2932736 to 3473408 tid 2 2024-09-15T01:44:35.587 INFO:tasks.rados.rados.0.smithi033.stdout:188: writing smithi03327189-35 from 3473408 to 4096000 tid 3 2024-09-15T01:44:35.587 INFO:tasks.rados.rados.0.smithi033.stdout:189: read oid 9 snap -1 2024-09-15T01:44:35.587 INFO:tasks.rados.rados.0.smithi033.stdout:189: expect (ObjNum 87 snap 10 seq_num 87) 2024-09-15T01:44:35.588 INFO:tasks.rados.rados.0.smithi033.stdout:190: read oid 22 snap -1 2024-09-15T01:44:35.588 INFO:tasks.rados.rados.0.smithi033.stdout:190: expect (ObjNum 92 snap 11 seq_num 92) 2024-09-15T01:44:35.588 INFO:tasks.rados.rados.0.smithi033.stdout:191: read oid 45 snap -1 2024-09-15T01:44:35.588 INFO:tasks.rados.rados.0.smithi033.stdout:191: expect deleted 2024-09-15T01:44:35.589 INFO:tasks.rados.rados.0.smithi033.stdout:192: rollback oid 47 current snap is 12 2024-09-15T01:44:35.589 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 47 to 11 2024-09-15T01:44:35.589 INFO:tasks.rados.rados.0.smithi033.stdout:193: snap_create 2024-09-15T01:44:35.593 INFO:tasks.rados.rados.0.smithi033.stdout:188: finishing write tid 1 to smithi03327189-35 2024-09-15T01:44:35.601 INFO:tasks.rados.rados.0.smithi033.stdout:192: finishing rollback tid 0 to smithi03327189-47 2024-09-15T01:44:35.601 INFO:tasks.rados.rados.0.smithi033.stdout:188: finishing write tid 2 to smithi03327189-35 2024-09-15T01:44:35.601 INFO:tasks.rados.rados.0.smithi033.stdout:192: finishing rollback tid 1 to smithi03327189-47 2024-09-15T01:44:35.607 INFO:tasks.rados.rados.0.smithi033.stdout:192: finishing rollback tid 2 to smithi03327189-47 2024-09-15T01:44:35.608 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 75 (ObjNum 86 snap 10 seq_num 86) dirty exists 2024-09-15T01:44:35.608 INFO:tasks.rados.rados.0.smithi033.stdout:188: finishing write tid 3 to smithi03327189-35 2024-09-15T01:44:35.608 INFO:tasks.rados.rados.0.smithi033.stdout:188: finishing write tid 4 to smithi03327189-35 2024-09-15T01:44:35.608 INFO:tasks.rados.rados.0.smithi033.stdout:190: expect (ObjNum 92 snap 11 seq_num 92) 2024-09-15T01:44:35.647 INFO:tasks.rados.rados.0.smithi033.stdout:188: finishing write tid 5 to smithi03327189-35 2024-09-15T01:44:35.648 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 41 (ObjNum 95 snap 12 seq_num 95) dirty exists 2024-09-15T01:44:35.648 INFO:tasks.rados.rados.0.smithi033.stdout:188: left oid 35 (ObjNum 95 snap 12 seq_num 95) 2024-09-15T01:44:35.648 INFO:tasks.rados.rados.0.smithi033.stdout:189: expect (ObjNum 87 snap 10 seq_num 87) 2024-09-15T01:44:35.656 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:35.651+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:35.657 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.657 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:35.666 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:35.757 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:35.752+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:35.765 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:35.765 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:35.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:35.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:35.823 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:35.848 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:35.859 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:35.851+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:35.866 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:35.874 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:35.874 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:35.875 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:35.875 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:35.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:35.888 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:35.908 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:35.929 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:35.952 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:35.959 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:35.952+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:35.981 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:35.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:36.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:36.027 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:36.060 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:36.056+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:36.065 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.592166109000001, 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.589734710999998, 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.589495895999999, 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.589464512999999, 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.610257083, 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.602342681, 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.602313418000001, 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.574876572000001, 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.573709826999998, 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.573336554000001, 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.572617456, 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082122610000000002, 2024-09-15T01:44:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545996+0000", 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545997+0000", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1999999999999999e-07 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546002+0000", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546013+0000", 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0954e-05 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546019+0000", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8089999999999998e-06 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554199+0000", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081794510000000008 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554208+0000", 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8959999999999998e-06 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.182 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.590402886, 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:36.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:36.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.602738099, 2024-09-15T01:44:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.600499638000001, 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:36.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.599929969, 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:36.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.206 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.599871001, 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=13, reqid=client.4632.0:302, at_version=43'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.598278123, 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094416429999999996, 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520336+0000", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520377+0000", 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1192000000000001e-05 2024-09-15T01:44:36.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520398+0000", 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1013e-05 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520411+0000", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.239e-05 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522362+0000", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001951803 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522402+0000", 2024-09-15T01:44:36.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9764000000000001e-05 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529721+0000", 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007318504 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529777+0000", 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6501e-05 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.584508833999999, 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083306100000000004, 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:36.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534105+0000", 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6000000000000004e-07 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534128+0000", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.315e-05 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534140+0000", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1508999999999999e-05 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534146+0000", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3230000000000003e-06 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534199+0000", 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3056999999999997e-05 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534209+0000", 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7999999999999993e-06 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542395+0000", 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081854519999999993 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542435+0000", 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0759000000000001e-05 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6119727929999996, 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:36.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:36.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:36.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6118790149999995, 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:36.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.236 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:36.238 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:36.232+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:36.339 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:36.332+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.390 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:36.390 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:36.390 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:36.440 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:36.435+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.500 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:36.500 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:36.500 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:36.500 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.507 INFO:tasks.rados.rados.0.smithi033.stdout:188: done (5 left) 2024-09-15T01:44:36.507 INFO:tasks.rados.rados.0.smithi033.stdout:189: done (4 left) 2024-09-15T01:44:36.508 INFO:tasks.rados.rados.0.smithi033.stdout:190: done (3 left) 2024-09-15T01:44:36.508 INFO:tasks.rados.rados.0.smithi033.stdout:191: done (2 left) 2024-09-15T01:44:36.508 INFO:tasks.rados.rados.0.smithi033.stdout:192: done (1 left) 2024-09-15T01:44:36.508 INFO:tasks.rados.rados.0.smithi033.stdout:193: done (0 left) 2024-09-15T01:44:36.509 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 45 current snap is 13 2024-09-15T01:44:36.509 INFO:tasks.rados.rados.0.smithi033.stdout:194: seq_num 96 ranges {0=458752,458752=688128,1146880=475136} 2024-09-15T01:44:36.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:36.516 INFO:tasks.rados.rados.0.smithi033.stdout:194: writing smithi03327189-45 from 0 to 458752 tid 1 2024-09-15T01:44:36.520 INFO:tasks.rados.rados.0.smithi033.stdout:194: writing smithi03327189-45 from 458752 to 1146880 tid 2 2024-09-15T01:44:36.524 INFO:tasks.rados.rados.0.smithi033.stdout:194: writing smithi03327189-45 from 1146880 to 1622016 tid 3 2024-09-15T01:44:36.524 INFO:tasks.rados.rados.0.smithi033.stdout:195: setattr oid 6 current snap is 13 2024-09-15T01:44:36.529 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 15 current snap is 13 2024-09-15T01:44:36.529 INFO:tasks.rados.rados.0.smithi033.stdout:196: seq_num 97 ranges {475136=475136,950272=442368,1392640=442368,1835008=278528} 2024-09-15T01:44:36.536 INFO:tasks.rados.rados.0.smithi033.stdout:196: writing smithi03327189-15 from 475136 to 950272 tid 1 2024-09-15T01:44:36.540 INFO:tasks.rados.rados.0.smithi033.stdout:196: writing smithi03327189-15 from 950272 to 1392640 tid 2 2024-09-15T01:44:36.541 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:36.535+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:36.544 INFO:tasks.rados.rados.0.smithi033.stdout:196: writing smithi03327189-15 from 1392640 to 1835008 tid 3 2024-09-15T01:44:36.546 INFO:tasks.rados.rados.0.smithi033.stdout:196: writing smithi03327189-15 from 1835008 to 2113536 tid 4 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: finishing write tid 1 to smithi03327189-45 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 29 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: finishing write tid 2 to smithi03327189-45 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: finishing write tid 3 to smithi03327189-45 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: finishing write tid 4 to smithi03327189-45 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: finishing write tid 5 to smithi03327189-45 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 45 v 79 (ObjNum 96 snap 13 seq_num 96) dirty exists 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: left oid 45 (ObjNum 96 snap 13 seq_num 96) 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:194: done (2 left) 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:195: done (1 left) 2024-09-15T01:44:36.547 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 18 current snap is 13 2024-09-15T01:44:36.548 INFO:tasks.rados.rados.0.smithi033.stdout:197: seq_num 98 ranges {1736704=491520,2228224=589824,2818048=557056,3375104=16384} 2024-09-15T01:44:36.560 INFO:tasks.rados.rados.0.smithi033.stdout:197: writing smithi03327189-18 from 1736704 to 2228224 tid 1 2024-09-15T01:44:36.565 INFO:tasks.rados.rados.0.smithi033.stdout:197: writing smithi03327189-18 from 2228224 to 2818048 tid 2 2024-09-15T01:44:36.569 INFO:tasks.rados.rados.0.smithi033.stdout:197: writing smithi03327189-18 from 2818048 to 3375104 tid 3 2024-09-15T01:44:36.570 INFO:tasks.rados.rados.0.smithi033.stdout:197: writing smithi03327189-18 from 3375104 to 3391488 tid 4 2024-09-15T01:44:36.570 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 13 current snap is 13 2024-09-15T01:44:36.570 INFO:tasks.rados.rados.0.smithi033.stdout:198: seq_num 99 ranges {98304=507904,606208=606208,1212416=557056} 2024-09-15T01:44:36.574 INFO:tasks.rados.rados.0.smithi033.stdout:198: writing smithi03327189-13 from 98304 to 606208 tid 1 2024-09-15T01:44:36.579 INFO:tasks.rados.rados.0.smithi033.stdout:198: writing smithi03327189-13 from 606208 to 1212416 tid 2 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:198: writing smithi03327189-13 from 1212416 to 1769472 tid 3 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 9 current snap is 13 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 1 to smithi03327189-15 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 2 to smithi03327189-15 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 3 to smithi03327189-15 2024-09-15T01:44:36.584 INFO:tasks.rados.rados.0.smithi033.stdout:199: seq_num 100 ranges {2965504=524288,3489792=753664,4243456=409600} 2024-09-15T01:44:36.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:44:36.605 INFO:tasks.rados.rados.0.smithi033.stdout:199: writing smithi03327189-9 from 2965504 to 3489792 tid 1 2024-09-15T01:44:36.611 INFO:tasks.rados.rados.0.smithi033.stdout:199: writing smithi03327189-9 from 3489792 to 4243456 tid 2 2024-09-15T01:44:36.614 INFO:tasks.rados.rados.0.smithi033.stdout:199: writing smithi03327189-9 from 4243456 to 4653056 tid 3 2024-09-15T01:44:36.615 INFO:tasks.rados.rados.0.smithi033.stdout:200: read oid 16 snap -1 2024-09-15T01:44:36.615 INFO:tasks.rados.rados.0.smithi033.stdout:200: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:44:36.615 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 24 current snap is 13 2024-09-15T01:44:36.615 INFO:tasks.rados.rados.0.smithi033.stdout:201: seq_num 101 ranges {1523712=540672,2064384=507904,2572288=622592,3194880=32768} 2024-09-15T01:44:36.628 INFO:tasks.rados.rados.0.smithi033.stdout:201: writing smithi03327189-24 from 1523712 to 2064384 tid 1 2024-09-15T01:44:36.631 INFO:tasks.rados.rados.0.smithi033.stdout:201: writing smithi03327189-24 from 2064384 to 2572288 tid 2 2024-09-15T01:44:36.637 INFO:tasks.rados.rados.0.smithi033.stdout:201: writing smithi03327189-24 from 2572288 to 3194880 tid 3 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:201: writing smithi03327189-24 from 3194880 to 3227648 tid 4 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:202: read oid 4 snap -1 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:202: expect deleted 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 49 current snap is 13 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 4 to smithi03327189-15 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 5 to smithi03327189-15 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:196: finishing write tid 6 to smithi03327189-15 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 20 (ObjNum 97 snap 13 seq_num 97) dirty exists 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:196: left oid 15 (ObjNum 97 snap 13 seq_num 97) 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 1 to smithi03327189-18 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 2 to smithi03327189-18 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 3 to smithi03327189-18 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 4 to smithi03327189-18 2024-09-15T01:44:36.638 INFO:tasks.rados.rados.0.smithi033.stdout:203: seq_num 102 ranges {0=557056,557056=638976,1196032=524288} 2024-09-15T01:44:36.641 INFO:tasks.rados.rados.0.smithi033.stdout:203: writing smithi03327189-49 from 0 to 557056 tid 1 2024-09-15T01:44:36.642 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:36.635+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:36.646 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.121336099000001, 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.121297093999999, 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.12126615, 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.106875031000001, 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083267830000000008, 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535447+0000", 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535448+0000", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9999999999999996e-07 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535466+0000", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8291000000000002e-05 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535476+0000", 2024-09-15T01:44:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8209999999999997e-06 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535482+0000", 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7919999999999998e-06 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540773+0000", 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052908319999999997 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540782+0000", 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6229999999999992e-06 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543747+0000", 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029650520000000001 2024-09-15T01:44:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543774+0000", 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6571999999999999e-05 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.126381850000001, 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.124386329, 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.124316796999999, 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:36.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:36.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.124294260999999, 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.690 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.113999465999999, 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071464149999999997, 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528323+0000", 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.06e-07 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528345+0000", 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1920999999999999e-05 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528355+0000", 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4830000000000005e-06 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528360+0000", 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5029999999999997e-06 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528430+0000", 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9640000000000004e-05 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528440+0000", 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0611999999999999e-05 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535446+0000", 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007005173 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535469+0000", 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3577000000000001e-05 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.124227069, 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:36.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:36.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.124157543999999, 2024-09-15T01:44:36.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.708 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:36.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:36.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:36.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:36.717 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.123915466, 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.719 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:36.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:36.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:36.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.123862032, 2024-09-15T01:44:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.123789853, 2024-09-15T01:44:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.109828295, 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:36.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:36.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:36.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:36.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:36.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.530055+0000", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.112266819, 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083933759999999993, 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530055+0000", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530055+0000", 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530057+0000", 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1909999999999999e-06 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530264+0000", 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000206956 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530280+0000", 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6765999999999998e-05 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530290+0000", 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2560000000000008e-06 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530858+0000", 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056869199999999998 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530872+0000", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4263999999999999e-05 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538414+0000", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075417289999999996 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538449+0000", 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4521999999999999e-05 2024-09-15T01:44:36.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:498 3.4s0 3:25aaa82b:::smithi03327189-9:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.038277361000000003, 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011653834, 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.758 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 498 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:36.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604049+0000", 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8739999999999997e-06 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604468+0000", 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041931899999999997 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604508+0000", 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9677e-05 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604524+0000", 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6030000000000001e-05 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604637+0000", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113602 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604816+0000", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606494+0000", 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016776149999999999 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615523+0000", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0090292470000000007 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615637+0000", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011338200000000001 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615699+0000", 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1970999999999993e-05 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:499 3.4s0 3:25aaa82b:::smithi03327189-9:head [append 0~753664] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.609832+0000", 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.032490131999999998, 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095746349999999997, 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 499 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.764 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.609832+0000", 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.609832+0000", 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.609835+0000", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8059999999999999e-06 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.610392+0000", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00055678499999999996 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.610413+0000", 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1719e-05 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.610421+0000", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7200000000000006e-06 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.610489+0000", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8124e-05 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.610619+0000", 2024-09-15T01:44:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012974300000000001 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.612579+0000", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019600070000000002 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619255+0000", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066765169999999999 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619330+0000", 2024-09-15T01:44:36.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4367000000000001e-05 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619407+0000", 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6847000000000005e-05 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:501 3.4s0 3:25aaa82b:::smithi03327189-9:head [setxattr _header (56) in=7b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.614425+0000", 2024-09-15T01:44:36.769 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.027897285000000001, 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0086410949999999997, 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 501 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614425+0000", 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614425+0000", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614426+0000", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2839999999999999e-06 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614434+0000", 2024-09-15T01:44:36.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5700000000000004e-06 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614454+0000", 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0550000000000001e-05 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614463+0000", 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0459999999999994e-06 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.618404+0000", 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039405159999999998 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.618529+0000", 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000124836 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.618969+0000", 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044051099999999998 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.620435+0000", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001465493 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.622977+0000", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002542037 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.623066+0000", 2024-09-15T01:44:36.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9252000000000005e-05 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:502 3.4s0 3:25aaa82b:::smithi03327189-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.027849540999999998, 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010446636, 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:36.775 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 502 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614474+0000", 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0020000000000001e-06 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614481+0000", 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9920000000000001e-06 2024-09-15T01:44:36.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614501+0000", 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0412e-05 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614509+0000", 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9920000000000007e-06 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619531+0000", 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050217589999999998 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619623+0000", 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2584000000000007e-05 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.623447+0000", 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038235180000000001 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.623538+0000", 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1099000000000003e-05 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.624919+0000", 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013812780000000001 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:36.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:36.781 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.781 INFO:tasks.rados.rados.0.smithi033.stdout:203: writing smithi03327189-49 from 557056 to 1196032 tid 2 2024-09-15T01:44:36.781 INFO:tasks.rados.rados.0.smithi033.stdout:203: writing smithi03327189-49 from 1196032 to 1720320 tid 3 2024-09-15T01:44:36.782 INFO:tasks.rados.rados.0.smithi033.stdout:196: done (7 left) 2024-09-15T01:44:36.782 INFO:tasks.rados.rados.0.smithi033.stdout:204: snap_remove snap 12 2024-09-15T01:44:36.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:36.784 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:36.779+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:36.885 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:36.880+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:36.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.888 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:36.888 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:36.888 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:36.986 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:36.980+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:36.989 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:36.989 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:36.989 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:36.989 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:36.989 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:36.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:37.087 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:37.080+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.557764470999999, 2024-09-15T01:44:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048467370000000003, 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535492+0000", 2024-09-15T01:44:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535493+0000", 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999996e-07 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535512+0000", 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9681e-05 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535531+0000", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8732e-05 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535539+0000", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9629999999999998e-06 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535593+0000", 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3452999999999998e-05 2024-09-15T01:44:37.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535619+0000", 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6557999999999999e-05 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540309+0000", 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046894509999999999 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540339+0000", 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0295e-05 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.573999500999999, 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:37.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.107 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.573848572999999, 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.571246134999999, 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050197059999999996, 2024-09-15T01:44:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522010+0000", 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522012+0000", 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.122e-06 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522045+0000", 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3034999999999998e-05 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522067+0000", 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2742e-05 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.522078+0000", 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0278000000000001e-05 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524135+0000", 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020572450000000001 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524153+0000", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8510000000000001e-05 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527017+0000", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028639719999999998 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527030+0000", 2024-09-15T01:44:37.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2802e-05 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.571513976999999, 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:37.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=34, reqid=client.4632.0:252, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.568990681999999, 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056439819999999996, 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.124 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524309+0000", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2670000000000003e-05 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524323+0000", 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3813e-05 2024-09-15T01:44:37.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524331+0000", 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2300000000000008e-06 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524482+0000", 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015112700000000001 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524497+0000", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4666e-05 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529866+0000", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053691579999999997 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529910+0000", 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3990999999999999e-05 2024-09-15T01:44:37.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s4 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.560412554999999, 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046971579999999999, 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532844+0000", 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532845+0000", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5600000000000005e-07 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532856+0000", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1635e-05 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532868+0000", 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1712000000000001e-05 2024-09-15T01:44:37.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532874+0000", 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3579999999999999e-06 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533006+0000", 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013125800000000001 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533016+0000", 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537501+0000", 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044853799999999997 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.537541+0000", 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790999999999999e-05 2024-09-15T01:44:37.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=135, reqid=client.4632.0:304, at_version=43'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.544222479, 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059241329999999998, 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549410+0000", 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000375464 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549431+0000", 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713999999999999e-05 2024-09-15T01:44:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549438+0000", 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549499+0000", 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1303999999999994e-05 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549514+0000", 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4901e-05 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554929+0000", 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054145510000000001 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554958+0000", 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9393000000000001e-05 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.138 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=136, reqid=client.4632.0:305, at_version=43'13, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.539745861, 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043787469999999997, 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553511+0000", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.553519+0000", 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2460000000000003e-06 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554227+0000", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00070824899999999999 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554241+0000", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4096e-05 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554249+0000", 2024-09-15T01:44:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6739999999999997e-06 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554324+0000", 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5050999999999997e-05 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554335+0000", 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0721e-05 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557866+0000", 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003531621 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.557890+0000", 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3088999999999999e-05 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=137, reqid=client.4632.0:306, at_version=43'14, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:37.143 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.53811923, 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046968319999999997, 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555137+0000", 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555138+0000", 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8599999999999998e-07 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555169+0000", 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0957000000000002e-05 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555181+0000", 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1789e-05 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555188+0000", 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7479999999999996e-06 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555224+0000", 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5938e-05 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.555230+0000", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.998e-06 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559806+0000", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045767990000000003 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.559834+0000", 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7917000000000001e-05 2024-09-15T01:44:37.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.8.17:1 3.es0 3:720ccbad:::smithi03327189-29:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.53367831, 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047997630000000003, 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559578+0000", 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559580+0000", 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.939e-06 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559588+0000", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4989999999999997e-06 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559605+0000", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559614+0000", 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6410000000000008e-06 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559684+0000", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0072000000000003e-05 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.559886+0000", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020191600000000001 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.564378+0000", 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044920860000000002 2024-09-15T01:44:37.151 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.56401072500000005, 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:44:37.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.560842434, 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063209149999999999, 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532415+0000", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534495+0000", 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020793859999999999 2024-09-15T01:44:37.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534525+0000", 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0420999999999999e-05 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534538+0000", 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3022e-05 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534680+0000", 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014239299999999999 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534711+0000", 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0543999999999998e-05 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538705+0000", 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039944020000000002 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538735+0000", 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9815000000000002e-05 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.4s5 57/53 ECSubWrite(tid=98, reqid=client.4632.0:498, at_version=57'43, trim_to=0'0, roll_forward_to=55'41))", 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.606693+0000", 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.48656364000000002, 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051105209999999998, 2024-09-15T01:44:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606693+0000", 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606693+0000", 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606693+0000", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3799999999999998e-07 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606782+0000", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8622000000000006e-05 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606861+0000", 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9207000000000002e-05 2024-09-15T01:44:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606881+0000", 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9667999999999999e-05 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606998+0000", 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000117419 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.607016+0000", 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7164e-05 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.611758+0000", 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047425829999999999 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.611804+0000", 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5420000000000002e-05 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.4s5 57/53 ECSubWrite(tid=107, reqid=client.4632.0:509, at_version=57'51, trim_to=0'0, roll_forward_to=57'48))", 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.639411+0000", 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.45384565799999999, 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048969019999999999, 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639411+0000", 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639411+0000", 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639412+0000", 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8000000000000005e-07 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639444+0000", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2657999999999998e-05 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639465+0000", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0429999999999999e-05 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639474+0000", 2024-09-15T01:44:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7709999999999995e-06 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639538+0000", 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4651000000000002e-05 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639549+0000", 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1201e-05 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.644279+0000", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047296700000000001 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.644308+0000", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8741000000000001e-05 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:515 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~557056] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.45311715000000002, 2024-09-15T01:44:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058152500000000001, 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 515 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640141+0000", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.829e-06 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640648+0000", 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050664500000000001 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640663+0000", 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5172e-05 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640670+0000", 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2749999999999998e-06 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640726+0000", 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5825000000000002e-05 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640828+0000", 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101492 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.642266+0000", 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014377940000000001 2024-09-15T01:44:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645206+0000", 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002940645 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645866+0000", 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066034100000000005 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645955+0000", 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8232000000000005e-05 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.44787896399999999, 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:44:37.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:44:37.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:517 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.44380639500000002, 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062744919999999996, 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 517 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649452+0000", 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2350000000000002e-06 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649713+0000", 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026092800000000001 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649729+0000", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5889999999999999e-05 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649740+0000", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649784+0000", 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4493999999999998e-05 2024-09-15T01:44:37.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649889+0000", 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105344 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.651145+0000", 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001255649 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655608+0000", 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044630939999999999 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655660+0000", 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1365e-05 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655725+0000", 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5124999999999997e-05 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:518 3.es0 3:7de15fb4:::smithi03327189-49:head [setxattr _header (57) in=7b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.44347993899999999, 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063479770000000003, 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 518 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649778+0000", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.144e-06 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649784+0000", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8270000000000003e-06 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649794+0000", 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0713999999999999e-05 2024-09-15T01:44:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649800+0000", 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2429999999999996e-06 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652010+0000", 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002210231 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652118+0000", 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108608 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652627+0000", 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050885000000000004 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655730+0000", 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031025250000000001 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655776+0000", 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5812000000000001e-05 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656125+0000", 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034902300000000001 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.193 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.44345035399999999, 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:44:37.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.198 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.199 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:37.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:37.201 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:37.196+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:37.301 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:37.299+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:37.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:37.354 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:37.354 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:37.354 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:37.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:37.402 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:37.396+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:37.461 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:37.462 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:37.462 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:37.462 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:37.462 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:37.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:37.503 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:37.496+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:37.570 INFO:tasks.rados.rados.0.smithi033.stdout:204: done (7 left) 2024-09-15T01:44:37.570 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 39 current snap is 13 2024-09-15T01:44:37.570 INFO:tasks.rados.rados.0.smithi033.stdout:205: seq_num 103 ranges {1867776=573440,2441216=786432,3227648=376832} 2024-09-15T01:44:37.577 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:44:37.585 INFO:tasks.rados.rados.0.smithi033.stdout:205: writing smithi03327189-39 from 1867776 to 2441216 tid 1 2024-09-15T01:44:37.591 INFO:tasks.rados.rados.0.smithi033.stdout:205: writing smithi03327189-39 from 2441216 to 3227648 tid 2 2024-09-15T01:44:37.593 INFO:tasks.rados.rados.0.smithi033.stdout:205: writing smithi03327189-39 from 3227648 to 3604480 tid 3 2024-09-15T01:44:37.593 INFO:tasks.rados.rados.0.smithi033.stdout:206: rmattr oid 36 current snap is 13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 5 to smithi03327189-18 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:197: finishing write tid 6 to smithi03327189-18 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 85 (ObjNum 98 snap 13 seq_num 98) dirty exists 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:197: left oid 18 (ObjNum 98 snap 13 seq_num 98) 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: finishing write tid 1 to smithi03327189-13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: finishing write tid 2 to smithi03327189-13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: finishing write tid 3 to smithi03327189-13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: finishing write tid 4 to smithi03327189-13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: finishing write tid 5 to smithi03327189-13 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 13 v 30 (ObjNum 99 snap 13 seq_num 99) dirty exists 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:198: left oid 13 (ObjNum 99 snap 13 seq_num 99) 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:199: finishing write tid 1 to smithi03327189-9 2024-09-15T01:44:37.594 INFO:tasks.rados.rados.0.smithi033.stdout:199: finishing write tid 2 to smithi03327189-9 2024-09-15T01:44:37.595 INFO:tasks.rados.rados.0.smithi033.stdout:200: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T01:44:37.604 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:37.600+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:37.606 INFO:tasks.rados.rados.0.smithi033.stdout:199: finishing write tid 3 to smithi03327189-9 2024-09-15T01:44:37.606 INFO:tasks.rados.rados.0.smithi033.stdout:199: finishing write tid 4 to smithi03327189-9 2024-09-15T01:44:37.606 INFO:tasks.rados.rados.0.smithi033.stdout:199: finishing write tid 5 to smithi03327189-9 2024-09-15T01:44:37.606 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 9 v 46 (ObjNum 100 snap 13 seq_num 100) dirty exists 2024-09-15T01:44:37.606 INFO:tasks.rados.rados.0.smithi033.stdout:199: left oid 9 (ObjNum 100 snap 13 seq_num 100) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 1 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:197: done (8 left) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:198: done (7 left) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:199: done (6 left) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:200: done (5 left) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:206: done (4 left) 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 31 current snap is 13 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 2 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 3 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 4 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:203: finishing write tid 1 to smithi03327189-49 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 5 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:201: finishing write tid 6 to smithi03327189-24 2024-09-15T01:44:37.607 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 52 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:201: left oid 24 (ObjNum 101 snap 13 seq_num 101) 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:203: finishing write tid 2 to smithi03327189-49 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:203: finishing write tid 3 to smithi03327189-49 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:203: finishing write tid 4 to smithi03327189-49 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:203: finishing write tid 5 to smithi03327189-49 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 49 v 13 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:203: left oid 49 (ObjNum 102 snap 13 seq_num 102) 2024-09-15T01:44:37.608 INFO:tasks.rados.rados.0.smithi033.stdout:207: seq_num 104 ranges {1376256=589824,1966080=540672,2506752=622592} 2024-09-15T01:44:37.616 INFO:tasks.rados.rados.0.smithi033.stdout:207: writing smithi03327189-31 from 1376256 to 1966080 tid 1 2024-09-15T01:44:37.620 INFO:tasks.rados.rados.0.smithi033.stdout:207: writing smithi03327189-31 from 1966080 to 2506752 tid 2 2024-09-15T01:44:37.625 INFO:tasks.rados.rados.0.smithi033.stdout:207: writing smithi03327189-31 from 2506752 to 3129344 tid 3 2024-09-15T01:44:37.625 INFO:tasks.rados.rados.0.smithi033.stdout:201: done (4 left) 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:202: done (3 left) 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:203: done (2 left) 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:208: copy_from oid 4 from oid 26 current snap is 13 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:209: rollback oid 11 current snap is 13 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 11 to 11 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:210: copy_from oid 7 from oid 6 current snap is 13 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:211: delete oid 38 current snap is 13 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:205: finishing write tid 1 to smithi03327189-39 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:205: finishing write tid 2 to smithi03327189-39 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:205: finishing write tid 3 to smithi03327189-39 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:205: finishing write tid 4 to smithi03327189-39 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:205: finishing write tid 5 to smithi03327189-39 2024-09-15T01:44:37.626 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 39 v 36 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-09-15T01:44:37.627 INFO:tasks.rados.rados.0.smithi033.stdout:205: left oid 39 (ObjNum 103 snap 13 seq_num 103) 2024-09-15T01:44:37.627 INFO:tasks.rados.rados.0.smithi033.stdout:207: finishing write tid 1 to smithi03327189-31 2024-09-15T01:44:37.630 INFO:tasks.rados.rados.0.smithi033.stdout:207: finishing write tid 2 to smithi03327189-31 2024-09-15T01:44:37.633 INFO:tasks.rados.rados.0.smithi033.stdout:205: done (5 left) 2024-09-15T01:44:37.633 INFO:tasks.rados.rados.0.smithi033.stdout:211: done (4 left) 2024-09-15T01:44:37.634 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 8 current snap is 13 2024-09-15T01:44:37.634 INFO:tasks.rados.rados.0.smithi033.stdout:212: seq_num 106 ranges {0=622592,622592=442368,1064960=524288,1589248=196608} 2024-09-15T01:44:37.641 INFO:tasks.rados.rados.0.smithi033.stdout:212: writing smithi03327189-8 from 0 to 622592 tid 1 2024-09-15T01:44:37.641 INFO:tasks.rados.rados.0.smithi033.stdout:212: writing smithi03327189-8 from 622592 to 1064960 tid 2 2024-09-15T01:44:37.642 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:37.642 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.128775082000001, 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:37.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.128720843, 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:37.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.128701705000001, 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:37.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:206 3.2s0 3:47035a48:::smithi03327189-20:head [append 0~606208] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.118382893, 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062804979999999998, 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 206 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519791+0000", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.519793+0000", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0190000000000001e-06 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520167+0000", 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037447000000000001 2024-09-15T01:44:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520180+0000", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3101e-05 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520186+0000", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9989999999999997e-06 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521198+0000", 2024-09-15T01:44:37.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001011807 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521308+0000", 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110059 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.522439+0000", 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011307229999999999 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.525693+0000", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003254159 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526006+0000", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000312971 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526071+0000", 2024-09-15T01:44:37.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5190000000000004e-05 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:209 3.2s0 3:47035a48:::smithi03327189-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.116391241999999, 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079111819999999992, 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 209 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521782+0000", 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521783+0000", 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9700000000000002e-07 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521784+0000", 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.736e-06 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521791+0000", 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0849999999999999e-06 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521803+0000", 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1345999999999999e-05 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526224+0000", 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004421431 2024-09-15T01:44:37.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526350+0000", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012618500000000001 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527766+0000", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014151299999999999 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.527853+0000", 2024-09-15T01:44:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7435999999999994e-05 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529693+0000", 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001840436 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0, backfill_or_async_recovery))", 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.105998692, 2024-09-15T01:44:37.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083125319999999992, 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532176+0000", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2191000000000001e-05 2024-09-15T01:44:37.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6229999999999999e-06 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532292+0000", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3135999999999999e-05 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532304+0000", 2024-09-15T01:44:37.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2481e-05 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540422+0000", 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081175480000000005 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540487+0000", 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5278999999999996e-05 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 33/32 ECSubWrite(tid=65, reqid=client.4632.0:199, at_version=0'0, trim_to=0'0, roll_forward_to=33'22))", 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.102899583999999, 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077245059999999999, 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535274+0000", 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535275+0000", 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000001e-07 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535296+0000", 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1590000000000002e-05 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535309+0000", 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2247e-05 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535315+0000", 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7870000000000004e-06 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535399+0000", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3398000000000003e-05 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535410+0000", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542978+0000", 2024-09-15T01:44:37.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075673989999999998 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542998+0000", 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0469999999999999e-05 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 38/34 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=38'34, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.117522406999999, 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065983789999999997, 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:37.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520651+0000", 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520652+0000", 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0330000000000001e-06 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520685+0000", 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3210999999999998e-05 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520702+0000", 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6880000000000001e-05 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520712+0000", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0003000000000001e-05 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520819+0000", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106947 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520835+0000", 2024-09-15T01:44:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5647000000000001e-05 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527222+0000", 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063877339999999999 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527249+0000", 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6923999999999998e-05 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.119726805000001, 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:37.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.686 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.119659143, 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.119633402, 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:37.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.106562593, 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.105520688, 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:37.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.104699883, 2024-09-15T01:44:37.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:37.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:37.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.5s1 52/51 ECSubReadReply(tid=96, attrs_read=0))", 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0947244129999998, 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077934340000000001, 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543449+0000", 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543464+0000", 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4994000000000001e-05 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.543468+0000", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.002e-06 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.547336+0000", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003867919 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:31.551242+0000", 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039065189999999998 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.107824025, 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084277399999999995, 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530350+0000", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530476+0000", 2024-09-15T01:44:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012622100000000001 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530495+0000", 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8326e-05 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530506+0000", 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1063999999999999e-05 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530816+0000", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030996700000000002 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530831+0000", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5129e-05 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538759+0000", 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079275890000000005 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538777+0000", 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.1055237849999999, 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089560869999999997, 2024-09-15T01:44:37.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532651+0000", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5239999999999999e-06 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532968+0000", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031664399999999999 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532988+0000", 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0676e-05 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533000+0000", 2024-09-15T01:44:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1318000000000001e-05 2024-09-15T01:44:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533071+0000", 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0779999999999997e-05 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533082+0000", 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1447000000000001e-05 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541581+0000", 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084990299999999994 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541606+0000", 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4668000000000002e-05 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:481 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~475136] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.102736672, 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095211319999999999, 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 481 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535439+0000", 2024-09-15T01:44:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5320000000000001e-06 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535767+0000", 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032738100000000002 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535789+0000", 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1917e-05 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535882+0000", 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3171000000000006e-05 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535944+0000", 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2124000000000004e-05 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.536068+0000", 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012412600000000001 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.537727+0000", 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016587010000000001 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544825+0000", 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070983599999999997 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544893+0000", 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8188999999999994e-05 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544958+0000", 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4630999999999998e-05 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:482 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~442368] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.099025683, 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087432670000000007, 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 482 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539150+0000", 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1459999999999999e-06 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539423+0000", 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000273195 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539441+0000", 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7725999999999999e-05 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539449+0000", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7700000000000001e-06 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539516+0000", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7158999999999999e-05 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539662+0000", 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000145861 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.540574+0000", 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091271500000000003 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544826+0000", 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042521360000000001 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547817+0000", 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029910129999999998 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547891+0000", 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3546000000000004e-05 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:486 3.9s1 3:9e656ec5:::smithi03327189-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.545137+0000", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0930361900000001, 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066456270000000003, 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 486 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545137+0000", 2024-09-15T01:44:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545137+0000", 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545138+0000", 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4299999999999998e-07 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545140+0000", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.933e-06 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545146+0000", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2729999999999999e-06 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.545149+0000", 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3639999999999999e-06 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547212+0000", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020625489999999999 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547294+0000", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2206999999999994e-05 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.550526+0000", 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032321799999999999 2024-09-15T01:44:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.550610+0000", 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3324000000000004e-05 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.551783+0000", 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011733539999999999 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:37.741 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:37.742 INFO:tasks.rados.rados.0.smithi033.stdout:212: writing smithi03327189-8 from 1064960 to 1589248 tid 3 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:212: writing smithi03327189-8 from 1589248 to 1785856 tid 4 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:213: read oid 34 snap -1 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:213: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:214: read oid 49 snap -1 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:214: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 21 current snap is 13 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:209: finishing rollback tid 0 to smithi03327189-11 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:215: seq_num 107 ranges {344064=638976,983040=458752,1441792=704512} 2024-09-15T01:44:37.743 INFO:tasks.rados.rados.0.smithi033.stdout:215: writing smithi03327189-21 from 344064 to 983040 tid 1 2024-09-15T01:44:37.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:215: writing smithi03327189-21 from 983040 to 1441792 tid 2 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:215: writing smithi03327189-21 from 1441792 to 2146304 tid 3 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:209: finishing rollback tid 1 to smithi03327189-11 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:209: finishing rollback tid 2 to smithi03327189-11 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 42 (ObjNum 88 snap 10 seq_num 88) dirty exists 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:210: finishing copy_from racing read to smithi03327189-7 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:207: finishing write tid 3 to smithi03327189-31 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:208: finishing copy_from racing read to smithi03327189-4 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:207: finishing write tid 4 to smithi03327189-31 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:209: done (7 left) 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 38 current snap is 13 2024-09-15T01:44:37.747 INFO:tasks.rados.rados.0.smithi033.stdout:207: finishing write tid 5 to smithi03327189-31 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 31 v 92 (ObjNum 104 snap 13 seq_num 104) dirty exists 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:207: left oid 31 (ObjNum 104 snap 13 seq_num 104) 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:210: finishing copy_from to smithi03327189-7 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 29 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:208: finishing copy_from to smithi03327189-4 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 44 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 1 to smithi03327189-8 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:216: seq_num 108 ranges {0=655360,655360=606208,1261568=475136,1736704=81920} 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:216: writing smithi03327189-38 from 0 to 655360 tid 1 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:216: writing smithi03327189-38 from 655360 to 1261568 tid 2 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:216: writing smithi03327189-38 from 1261568 to 1736704 tid 3 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:216: writing smithi03327189-38 from 1736704 to 1818624 tid 4 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:207: done (7 left) 2024-09-15T01:44:37.748 INFO:tasks.rados.rados.0.smithi033.stdout:208: done (6 left) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:210: done (5 left) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:217: delete oid 28 current snap is 13 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 2 to smithi03327189-8 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 3 to smithi03327189-8 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 4 to smithi03327189-8 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:213: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 5 to smithi03327189-8 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:214: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: finishing write tid 6 to smithi03327189-8 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 8 v 49 (ObjNum 106 snap 13 seq_num 106) dirty exists 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: left oid 8 (ObjNum 106 snap 13 seq_num 106) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:215: finishing write tid 1 to smithi03327189-21 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:212: done (5 left) 2024-09-15T01:44:37.749 INFO:tasks.rados.rados.0.smithi033.stdout:213: done (4 left) 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:214: done (3 left) 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:217: done (2 left) 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 29 current snap is 13 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:215: finishing write tid 2 to smithi03327189-21 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:215: finishing write tid 3 to smithi03327189-21 2024-09-15T01:44:37.750 INFO:tasks.rados.rados.0.smithi033.stdout:218: seq_num 110 ranges {475136=688128,1163264=507904,1671168=655360} 2024-09-15T01:44:37.751 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:37.735+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:37.751 INFO:tasks.rados.rados.0.smithi033.stdout:218: writing smithi03327189-29 from 475136 to 1163264 tid 1 2024-09-15T01:44:37.755 INFO:tasks.rados.rados.0.smithi033.stdout:218: writing smithi03327189-29 from 1163264 to 1671168 tid 2 2024-09-15T01:44:37.759 INFO:tasks.rados.rados.0.smithi033.stdout:218: writing smithi03327189-29 from 1671168 to 2326528 tid 3 2024-09-15T01:44:37.759 INFO:tasks.rados.rados.0.smithi033.stdout:219: snap_remove snap 11 2024-09-15T01:44:37.843 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:37.835+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:37.854 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:37.854 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:37.854 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:37.854 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:37.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:37.944 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:37.940+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:37.964 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:37.964 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:37.964 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:37.965 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:37.965 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:37.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:38.044 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:38.040+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.552697884000001, 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:44:38.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:44:38.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.552639538000001, 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:44:38.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.085 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.552560781, 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.087 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:44:38.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:44:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.55238078, 2024-09-15T01:44:38.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:44:38.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.5420708250000001, 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:44:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:44:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:44:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.48682507899999999, 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:44:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:44:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:44:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:522 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~786432] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.48129595600000002, 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078988010000000004, 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 522 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590056+0000", 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7269999999999999e-06 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590849+0000", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00079286600000000004 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590866+0000", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7288000000000001e-05 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590872+0000", 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7169999999999996e-06 2024-09-15T01:44:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590926+0000", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4212999999999997e-05 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.591041+0000", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115254 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592382+0000", 2024-09-15T01:44:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013405839999999999 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.596905+0000", 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045232959999999996 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597832+0000", 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000926299 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597953+0000", 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121557 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:523 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~376832] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.47870493200000003, 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080569660000000005, 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 523 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592647+0000", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0269999999999998e-06 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593297+0000", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00064929400000000002 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593319+0000", 2024-09-15T01:44:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2439e-05 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593329+0000", 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4010000000000003e-06 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593780+0000", 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00045140499999999999 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593878+0000", 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.7739999999999996e-05 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.594895+0000", 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0010174769999999999 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.596948+0000", 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0020530800000000001 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.600592+0000", 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036435949999999999 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.600702+0000", 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011050800000000001 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.47798300999999999, 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:44:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.44710338900000002, 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:44:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.44702402099999999, 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:538 3.0s0 3:0899e339:::smithi03327189-38:head [delete] snapc 19=[18] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.44691350099999999, 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080048380000000002, 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 538 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624438+0000", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2559999999999999e-06 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624442+0000", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9099999999999998e-06 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624456+0000", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4392e-05 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624465+0000", 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1629999999999996e-06 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625251+0000", 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00078665199999999999 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625564+0000", 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000313126 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.626639+0000", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0010743 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.631711+0000", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050726390000000003 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.632343+0000", 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00063175199999999996 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.632442+0000", 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8647999999999995e-05 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.42713363999999998, 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:44:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:542 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~196608] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.42599479600000001, 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0086966509999999997, 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 542 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645357+0000", 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.057e-06 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645555+0000", 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000198593 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645569+0000", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4181e-05 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645574+0000", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5750000000000002e-06 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.646955+0000", 2024-09-15T01:44:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013807369999999999 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.647078+0000", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012335100000000001 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.647826+0000", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00074837899999999997 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649457+0000", 2024-09-15T01:44:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001630514 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653983+0000", 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045256189999999998 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654052+0000", 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9645000000000002e-05 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.42577097800000002, 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:44:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:44:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.42575645299999998, 2024-09-15T01:44:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:44:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:547 3.0s0 3:08c156cd:::smithi03327189-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.42570803099999999, 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0091027969999999993, 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 547 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645643+0000", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.79e-07 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645644+0000", 2024-09-15T01:44:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.699e-06 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645650+0000", 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.942e-06 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645653+0000", 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.452e-06 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653028+0000", 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073755519999999996 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653094+0000", 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5718999999999999e-05 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654745+0000", 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016511539999999999 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.40337069199999998, 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:44:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:44:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:560 3.0s0 3:0899e339:::smithi03327189-38:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.399638243, 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090281700000000003, 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 560 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671713+0000", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671717+0000", 2024-09-15T01:44:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1749999999999998e-06 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671737+0000", 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.948e-05 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671744+0000", 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9569999999999996e-06 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676355+0000", 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046115949999999996 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676528+0000", 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017328999999999999 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677261+0000", 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073246900000000002 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677982+0000", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072084199999999997 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680660+0000", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002678541 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680740+0000", 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9963000000000007e-05 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.39960036999999998, 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:44:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:38.183 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:38.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:38.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:38.186 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:38.179+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:38.285 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:38.280+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:38.290 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:38.291 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:38.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:38.332 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:38.353 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:38.375 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:38.386 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:38.380+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:38.397 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:38.421 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:38.444 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:38.447 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.447 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:38.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:38.466 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:38.486 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:38.480+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:38.487 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:38.508 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:38.533 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:38.553 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.553 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:38.556 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:38.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:38.582 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:219: done (3 left) 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:220: read oid 24 snap -1 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:220: expect (ObjNum 101 snap 13 seq_num 101) 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:221: rollback oid 41 current snap is 13 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:221: done (4 left) 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:222: read oid 43 snap -1 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:222: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 7 current snap is 13 2024-09-15T01:44:38.583 INFO:tasks.rados.rados.0.smithi033.stdout:223: seq_num 111 ranges {98304=704512,802816=655360,1458176=507904} 2024-09-15T01:44:38.587 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:38.580+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:38.587 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:38.588 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:38.599 INFO:tasks.rados.rados.0.smithi033.stdout:223: writing smithi03327189-7 from 98304 to 802816 tid 1 2024-09-15T01:44:38.605 INFO:tasks.rados.rados.0.smithi033.stdout:223: writing smithi03327189-7 from 802816 to 1458176 tid 2 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:223: writing smithi03327189-7 from 1458176 to 1966080 tid 3 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:215: finishing write tid 4 to smithi03327189-21 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:215: finishing write tid 5 to smithi03327189-21 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 21 v 33 (ObjNum 107 snap 13 seq_num 107) dirty exists 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:215: left oid 21 (ObjNum 107 snap 13 seq_num 107) 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 1 to smithi03327189-38 2024-09-15T01:44:38.610 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 2 to smithi03327189-38 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 3 to smithi03327189-38 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 4 to smithi03327189-38 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 5 to smithi03327189-38 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:216: finishing write tid 6 to smithi03327189-38 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 54 (ObjNum 108 snap 13 seq_num 108) dirty exists 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:216: left oid 38 (ObjNum 108 snap 13 seq_num 108) 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: finishing write tid 1 to smithi03327189-29 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: finishing write tid 2 to smithi03327189-29 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: finishing write tid 3 to smithi03327189-29 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: finishing write tid 4 to smithi03327189-29 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: finishing write tid 5 to smithi03327189-29 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 29 v 18 (ObjNum 110 snap 13 seq_num 110) dirty exists 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:218: left oid 29 (ObjNum 110 snap 13 seq_num 110) 2024-09-15T01:44:38.611 INFO:tasks.rados.rados.0.smithi033.stdout:222: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:44:38.631 INFO:tasks.rados.rados.0.smithi033.stdout:215: done (5 left) 2024-09-15T01:44:38.631 INFO:tasks.rados.rados.0.smithi033.stdout:216: done (4 left) 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:218: done (3 left) 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:222: done (2 left) 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:224: read oid 32 snap -1 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:224: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:225: copy_from oid 19 from oid 36 current snap is 13 2024-09-15T01:44:38.632 INFO:tasks.rados.rados.0.smithi033.stdout:220: expect (ObjNum 101 snap 13 seq_num 101) 2024-09-15T01:44:38.672 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.672 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:38.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:38.687 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:38.679+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:38.754 INFO:tasks.rados.rados.0.smithi033.stdout:223: finishing write tid 1 to smithi03327189-7 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:223: finishing write tid 2 to smithi03327189-7 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:223: finishing write tid 3 to smithi03327189-7 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:223: finishing write tid 4 to smithi03327189-7 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:223: finishing write tid 5 to smithi03327189-7 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 40 (ObjNum 111 snap 13 seq_num 111) dirty exists 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:223: left oid 7 (ObjNum 111 snap 13 seq_num 111) 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:225: finishing copy_from racing read to smithi03327189-19 2024-09-15T01:44:38.755 INFO:tasks.rados.rados.0.smithi033.stdout:224: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:38.780 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.780 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:38.788 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:38.784+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:38.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:38.814 INFO:tasks.rados.rados.0.smithi033.stdout:225: finishing copy_from to smithi03327189-19 2024-09-15T01:44:38.814 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 19 v 12 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T01:44:38.814 INFO:tasks.rados.rados.0.smithi033.stdout:220: done (3 left) 2024-09-15T01:44:38.815 INFO:tasks.rados.rados.0.smithi033.stdout:223: done (2 left) 2024-09-15T01:44:38.815 INFO:tasks.rados.rados.0.smithi033.stdout:224: done (1 left) 2024-09-15T01:44:38.815 INFO:tasks.rados.rados.0.smithi033.stdout:225: done (0 left) 2024-09-15T01:44:38.815 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 20 current snap is 13 2024-09-15T01:44:38.815 INFO:tasks.rados.rados.0.smithi033.stdout:226: seq_num 112 ranges {983040=720896,1703936=802816,2506752=360448} 2024-09-15T01:44:38.833 INFO:tasks.rados.rados.0.smithi033.stdout:226: writing smithi03327189-20 from 983040 to 1703936 tid 1 2024-09-15T01:44:38.838 INFO:tasks.rados.rados.0.smithi033.stdout:226: writing smithi03327189-20 from 1703936 to 2506752 tid 2 2024-09-15T01:44:38.840 INFO:tasks.rados.rados.0.smithi033.stdout:226: writing smithi03327189-20 from 2506752 to 2867200 tid 3 2024-09-15T01:44:38.840 INFO:tasks.rados.rados.0.smithi033.stdout:227: snap_create 2024-09-15T01:44:38.842 INFO:tasks.rados.rados.0.smithi033.stdout:226: finishing write tid 1 to smithi03327189-20 2024-09-15T01:44:38.847 INFO:tasks.rados.rados.0.smithi033.stdout:226: finishing write tid 2 to smithi03327189-20 2024-09-15T01:44:38.849 INFO:tasks.rados.rados.0.smithi033.stdout:226: finishing write tid 3 to smithi03327189-20 2024-09-15T01:44:38.851 INFO:tasks.rados.rados.0.smithi033.stdout:226: finishing write tid 4 to smithi03327189-20 2024-09-15T01:44:38.853 INFO:tasks.rados.rados.0.smithi033.stdout:226: finishing write tid 5 to smithi03327189-20 2024-09-15T01:44:38.853 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 24 (ObjNum 112 snap 13 seq_num 112) dirty exists 2024-09-15T01:44:38.853 INFO:tasks.rados.rados.0.smithi033.stdout:226: left oid 20 (ObjNum 112 snap 13 seq_num 112) 2024-09-15T01:44:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.888 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:38.889 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:38.883+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:38.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:38.990 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:38.984+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:38.999 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:38.999 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.009 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:39.091 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:39.084+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:39.108 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.108 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:39.191 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:39.184+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:39.263 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.264 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:39.293 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:39.288+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:39.385 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.385 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.394 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:39.387+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:39.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:39.490 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.490 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.494 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:39.488+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:39.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:39.588 INFO:tasks.rados.rados.0.smithi033.stdout:226: done (1 left) 2024-09-15T01:44:39.588 INFO:tasks.rados.rados.0.smithi033.stdout:227: done (0 left) 2024-09-15T01:44:39.588 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 12 current snap is 14 2024-09-15T01:44:39.588 INFO:tasks.rados.rados.0.smithi033.stdout:228: seq_num 113 ranges {1622016=737280,2359296=557056,2916352=606208} 2024-09-15T01:44:39.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:44:39.595 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:39.587+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:39.597 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.598 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.607 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:39.609 INFO:tasks.rados.rados.0.smithi033.stdout:228: writing smithi03327189-12 from 1622016 to 2359296 tid 1 2024-09-15T01:44:39.612 INFO:tasks.rados.rados.0.smithi033.stdout:228: writing smithi03327189-12 from 2359296 to 2916352 tid 2 2024-09-15T01:44:39.615 INFO:tasks.rados.rados.0.smithi033.stdout:228: writing smithi03327189-12 from 2916352 to 3522560 tid 3 2024-09-15T01:44:39.616 INFO:tasks.rados.rados.0.smithi033.stdout:229: setattr oid 48 current snap is 14 2024-09-15T01:44:39.617 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 29 current snap is 14 2024-09-15T01:44:39.617 INFO:tasks.rados.rados.0.smithi033.stdout:230: seq_num 114 ranges {2326528=753664,3080192=688128,3768320=475136} 2024-09-15T01:44:39.635 INFO:tasks.rados.rados.0.smithi033.stdout:230: writing smithi03327189-29 from 2326528 to 3080192 tid 1 2024-09-15T01:44:39.640 INFO:tasks.rados.rados.0.smithi033.stdout:230: writing smithi03327189-29 from 3080192 to 3768320 tid 2 2024-09-15T01:44:39.644 INFO:tasks.rados.rados.0.smithi033.stdout:230: writing smithi03327189-29 from 3768320 to 4243456 tid 3 2024-09-15T01:44:39.644 INFO:tasks.rados.rados.0.smithi033.stdout:231: copy_from oid 10 from oid 45 current snap is 14 2024-09-15T01:44:39.644 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 48 v 44 (ObjNum 177 snap 0 seq_num 2491241200) dirty exists 2024-09-15T01:44:39.644 INFO:tasks.rados.rados.0.smithi033.stdout:228: finishing write tid 1 to smithi03327189-12 2024-09-15T01:44:39.644 INFO:tasks.rados.rados.0.smithi033.stdout:228: finishing write tid 2 to smithi03327189-12 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:228: finishing write tid 3 to smithi03327189-12 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:228: finishing write tid 4 to smithi03327189-12 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:228: finishing write tid 5 to smithi03327189-12 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 21 (ObjNum 113 snap 14 seq_num 113) dirty exists 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:228: left oid 12 (ObjNum 113 snap 14 seq_num 113) 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:228: done (3 left) 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:229: done (2 left) 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 39 current snap is 14 2024-09-15T01:44:39.645 INFO:tasks.rados.rados.0.smithi033.stdout:232: seq_num 115 ranges {3604480=770048,4374528=704512,5079040=458752} 2024-09-15T01:44:39.669 INFO:tasks.rados.rados.0.smithi033.stdout:232: writing smithi03327189-39 from 3604480 to 4374528 tid 1 2024-09-15T01:44:39.674 INFO:tasks.rados.rados.0.smithi033.stdout:232: writing smithi03327189-39 from 4374528 to 5079040 tid 2 2024-09-15T01:44:39.677 INFO:tasks.rados.rados.0.smithi033.stdout:232: writing smithi03327189-39 from 5079040 to 5537792 tid 3 2024-09-15T01:44:39.677 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 11 current snap is 14 2024-09-15T01:44:39.677 INFO:tasks.rados.rados.0.smithi033.stdout:230: finishing write tid 1 to smithi03327189-29 2024-09-15T01:44:39.677 INFO:tasks.rados.rados.0.smithi033.stdout:233: seq_num 116 ranges {2670592=786432,3457024=458752,3915776=442368,4358144=262144} 2024-09-15T01:44:39.696 INFO:tasks.rados.rados.0.smithi033.stdout:233: writing smithi03327189-11 from 2670592 to 3457024 tid 1 2024-09-15T01:44:39.696 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:39.688+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:39.699 INFO:tasks.rados.rados.0.smithi033.stdout:233: writing smithi03327189-11 from 3457024 to 3915776 tid 2 2024-09-15T01:44:39.702 INFO:tasks.rados.rados.0.smithi033.stdout:233: writing smithi03327189-11 from 3915776 to 4358144 tid 3 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:233: writing smithi03327189-11 from 4358144 to 4620288 tid 4 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:234: rollback oid 25 current snap is 14 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 25 to 13 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:235: read oid 47 snap 13 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:235: expect (ObjNum 86 snap 10 seq_num 86) 2024-09-15T01:44:39.704 INFO:tasks.rados.rados.0.smithi033.stdout:236: delete oid 30 current snap is 14 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:230: finishing write tid 2 to smithi03327189-29 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:230: finishing write tid 3 to smithi03327189-29 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:230: finishing write tid 4 to smithi03327189-29 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:230: finishing write tid 5 to smithi03327189-29 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 29 v 25 (ObjNum 114 snap 14 seq_num 114) dirty exists 2024-09-15T01:44:39.705 INFO:tasks.rados.rados.0.smithi033.stdout:230: left oid 29 (ObjNum 114 snap 14 seq_num 114) 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:231: finishing copy_from racing read to smithi03327189-10 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:231: finishing copy_from to smithi03327189-10 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 10 v 79 (ObjNum 96 snap 13 seq_num 96) dirty exists 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: finishing write tid 1 to smithi03327189-39 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: finishing write tid 2 to smithi03327189-39 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: finishing write tid 3 to smithi03327189-39 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: finishing write tid 4 to smithi03327189-39 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: finishing write tid 5 to smithi03327189-39 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 39 v 60 (ObjNum 115 snap 14 seq_num 115) dirty exists 2024-09-15T01:44:39.706 INFO:tasks.rados.rados.0.smithi033.stdout:232: left oid 39 (ObjNum 115 snap 14 seq_num 115) 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:230: done (6 left) 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:231: done (5 left) 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:232: done (4 left) 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:236: done (3 left) 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 7 current snap is 14 2024-09-15T01:44:39.710 INFO:tasks.rados.rados.0.smithi033.stdout:237: seq_num 118 ranges {1966080=442368,2408448=753664,3162112=655360,3817472=147456} 2024-09-15T01:44:39.714 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.714 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:39.724 INFO:tasks.rados.rados.0.smithi033.stdout:237: writing smithi03327189-7 from 1966080 to 2408448 tid 1 2024-09-15T01:44:39.729 INFO:tasks.rados.rados.0.smithi033.stdout:237: writing smithi03327189-7 from 2408448 to 3162112 tid 2 2024-09-15T01:44:39.733 INFO:tasks.rados.rados.0.smithi033.stdout:237: writing smithi03327189-7 from 3162112 to 3817472 tid 3 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:237: writing smithi03327189-7 from 3817472 to 3964928 tid 4 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:238: snap_create 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 1 to smithi03327189-11 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 2 to smithi03327189-11 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:234: finishing rollback tid 0 to smithi03327189-25 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 3 to smithi03327189-11 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:234: finishing rollback tid 1 to smithi03327189-25 2024-09-15T01:44:39.735 INFO:tasks.rados.rados.0.smithi033.stdout:235: expect (ObjNum 86 snap 10 seq_num 86) 2024-09-15T01:44:39.776 INFO:tasks.rados.rados.0.smithi033.stdout:234: finishing rollback tid 2 to smithi03327189-25 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 25 v 24 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 4 to smithi03327189-11 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 5 to smithi03327189-11 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:233: finishing write tid 6 to smithi03327189-11 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 50 (ObjNum 116 snap 14 seq_num 116) dirty exists 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:233: left oid 11 (ObjNum 116 snap 14 seq_num 116) 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 1 to smithi03327189-7 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 2 to smithi03327189-7 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 3 to smithi03327189-7 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 4 to smithi03327189-7 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 5 to smithi03327189-7 2024-09-15T01:44:39.777 INFO:tasks.rados.rados.0.smithi033.stdout:237: finishing write tid 6 to smithi03327189-7 2024-09-15T01:44:39.778 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 7 v 46 (ObjNum 118 snap 14 seq_num 118) dirty exists 2024-09-15T01:44:39.778 INFO:tasks.rados.rados.0.smithi033.stdout:237: left oid 7 (ObjNum 118 snap 14 seq_num 118) 2024-09-15T01:44:39.797 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:39.792+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:39.815 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.815 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:39.898 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:39.891+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:39.923 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:39.923 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:39.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:39.999 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:39.992+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:40.031 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.031 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:40.100 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:40.096+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:40.139 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.139 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.148 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:40.201 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:40.196+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.255 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:40.302 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:40.295+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:40.353 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.353 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.363 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:40.402 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:40.400+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:40.503 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:40.496+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:40.505 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.505 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.514 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:40.576 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:40.576 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:40.576 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T01:44:40.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T01:44:40.598 INFO:tasks.rados.rados.0.smithi033.stdout:233: done (4 left) 2024-09-15T01:44:40.598 INFO:tasks.rados.rados.0.smithi033.stdout:234: done (3 left) 2024-09-15T01:44:40.599 INFO:tasks.rados.rados.0.smithi033.stdout:235: done (2 left) 2024-09-15T01:44:40.599 INFO:tasks.rados.rados.0.smithi033.stdout:237: done (1 left) 2024-09-15T01:44:40.599 INFO:tasks.rados.rados.0.smithi033.stdout:238: done (0 left) 2024-09-15T01:44:40.599 INFO:tasks.rados.rados.0.smithi033.stdout:239: snap_remove snap 14 2024-09-15T01:44:40.601 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:44:40.604 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:40.596+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:40.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:40.619 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:40.619 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:40.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:40.704 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:40.700+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:40.749 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:40.765 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:40.766 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:40.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:40.775 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:40.796 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:40.805 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:40.800+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:40.816 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:40.839 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:40.863 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:40.868 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:40.868 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:40.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:40.886 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:40.887 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 61, 2024-09-15T01:44:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "d386169c-ec95-4076-a6ad-eaab3ccf750b", 2024-09-15T01:44:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-09-15T01:43:41.737992+0000", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-09-15T01:44:40.582883+0000", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "last_up_change": "2024-09-15T01:43:52.350040+0000", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "last_in_change": "2024-09-15T01:44:29.503950+0000", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_num": 5801984, 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_set": [ 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "noscrub", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "pglog_hardlimit", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "purged_snapdirs", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_deletes", 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "sortbitwise" 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_version": 6, 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T01:44:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_snapshot": "", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_max": 3, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd": 12, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "require_min_compat_client": "luminous", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min_compat_client": "luminous", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "require_osd_release": "quincy", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "pools": [ 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 1, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": ".mgr", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:52.349089+0000", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool", 2024-09-15T01:44:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 1, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 1, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 1, 2024-09-15T01:44:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 1, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 1, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "20", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 0, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 0, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "options": { 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_max": 32, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_min": 1 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": {} 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 2, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "rbd", 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:54.322526+0000", 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 8, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 8, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 8, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 8, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 8, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "23", 2024-09-15T01:44:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 3, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 23, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd": {} 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "unique_pool_0", 2024-09-15T01:44:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:44:05.562069+0000", 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 3, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 8, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 5, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 1, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 16, 2024-09-15T01:44:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 16, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 16, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 16, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 16, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "61", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:44:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 29, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 61, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:44:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "lrcprofile", 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:44:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 16384, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "rados": {} 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "5bbf5b49-bace-424f-b96c-91c3a25e48e0", 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-09-15T01:44:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6802", 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6803", 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6806", 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6807", 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6804", 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6805", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6801/22087", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6803/22087", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6807/22087", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6805/22087", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464", 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6808", 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6809", 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6810", 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6811", 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6814", 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6815", 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6812", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6813", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6809/22086", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6811/22086", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6815/22086", 2024-09-15T01:44:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6813/22086", 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "fcc939b7-c3fb-4117-af41-9eec226f3020", 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 50, 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6816", 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6817", 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6818", 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6819", 2024-09-15T01:44:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6822", 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6823", 2024-09-15T01:44:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6820", 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6821", 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:44:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6817/22088", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6819/22088", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6823/22088", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6821/22088", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "57f1d668-f85b-4b0c-9093-fb5fef4b91ef", 2024-09-15T01:44:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6824", 2024-09-15T01:44:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6825", 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6826", 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6827", 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6830", 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6831", 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6828", 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6829", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6825/22089", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6827/22089", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6831/22089", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6829/22089", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "a274d12c-8683-491d-8656-5af5d12164ee", 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6800", 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6801", 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6802", 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6803", 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6806", 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6807", 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6804", 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6805", 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:44:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6801/22096", 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6803/22096", 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6807/22096", 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6805/22096", 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:44:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "f0a90a47-9b16-46f6-a2a7-afa4a6f2a298", 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6816", 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6817", 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6818", 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6819", 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6822", 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6823", 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6820", 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6821", 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6817/22097", 2024-09-15T01:44:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6819/22097", 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6823/22097", 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6821/22097", 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "194e23d1-7f52-43ee-8557-52d75a3c22dc", 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 27, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6808", 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6809", 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6810", 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6811", 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6814", 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6815", 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6812", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6813", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6809/22108", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6811/22108", 2024-09-15T01:44:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6815/22108", 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6813/22108", 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1d1c61e7-be03-4c72-bc95-e187eb95a1b8", 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 18, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6824", 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6825", 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6826", 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6827", 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6830", 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6831", 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6828", 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6829", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6825/22109", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6827/22109", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6831/22109", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6829/22109", 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1f23463d-2214-416b-b3ba-23132f2d84ee", 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 0, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 52, 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6801", 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6803", 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6805", 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6807", 2024-09-15T01:44:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6813", 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6815", 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6809", 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6811", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6803/22357", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6807/22357", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6815/22357", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6811/22357", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "39226fc7-a654-41c6-9d70-5ea2f69c236b", 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 53, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6800", 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6802", 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6804", 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6806", 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6812", 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6814", 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6808", 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6810", 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6802/22359", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6806/22359", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6814/22359", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6810/22359", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "eaafc17e-407d-43aa-8083-2c56660d7db4", 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 0, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6816", 2024-09-15T01:44:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6817", 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6818", 2024-09-15T01:44:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6819", 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6822", 2024-09-15T01:44:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6823", 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6820", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6821", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6817/22358", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6819/22358", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6823/22358", 2024-09-15T01:44:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6821/22358", 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "05fdd01d-8c78-4a61-81e7-1d9816e951a3", 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 0, 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-09-15T01:44:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 27, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6824", 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6825", 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6826", 2024-09-15T01:44:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6827", 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6830", 2024-09-15T01:44:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6831", 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6828", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6829", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6825/22360", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6827/22360", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6831/22360", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6829/22360", 2024-09-15T01:44:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_xinfo": [ 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.101167+0000", 2024-09-15T01:44:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.166075+0000", 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.012320+0000", 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.227835+0000", 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.010828+0000", 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 65536, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.170791+0000", 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.037098+0000", 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.281668+0000", 2024-09-15T01:44:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:49.661197+0000", 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.009199+0000", 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.143021+0000", 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:44:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 65536, 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.187329+0000", 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap": [ 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.e", 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: 11, 2024-09-15T01:44:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: 9, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 5, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 6, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 10, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap_items": [], 2024-09-15T01:44:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_temp": [], 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_temp": [], 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklist": {}, 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "range_blocklist": {}, 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profiles": { 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default": { 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "2", 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "1", 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "jerasure", 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "technique": "reed_sol_van" 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "lrcprofile": { 2024-09-15T01:44:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-device-class": "", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-root": "default", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "4", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "l": "3", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "2", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "lrcprofile", 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "lrc" 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps_queue": [ 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "snaps": [ 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 19, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 1 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 21, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 1 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 24, 2024-09-15T01:44:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 2 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "new_removed_snaps": [], 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "new_purged_snaps": [], 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_node_flags": {}, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "device_class_flags": {}, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": { 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_enabled": false, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_bucket_count": 0, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "degraded_stretch_mode": 0, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "recovering_stretch_mode": 0, 2024-09-15T01:44:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_bucket": 0 2024-09-15T01:44:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.987 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:40.989 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': 'd386169c-ec95-4076-a6ad-eaab3ccf750b', 'created': '2024-09-15T01:43:41.737992+0000', 'modified': '2024-09-15T01:44:40.582883+0000', 'last_up_change': '2024-09-15T01:43:52.350040+0000', 'last_in_change': '2024-09-15T01:44:29.503950+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:43:52.349089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:43:54.322526+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T01:44:05.562069+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 8, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'lrcprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '5bbf5b49-bace-424f-b96c-91c3a25e48e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 22087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 22087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 22087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6805', 'nonce': 22087}]}, 'public_addr': '172.21.15.33:6801/22087', 'cluster_addr': '172.21.15.33:6803/22087', 'heartbeat_back_addr': '172.21.15.33:6807/22087', 'heartbeat_front_addr': '172.21.15.33:6805/22087', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6808', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6809', 'nonce': 22086}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6810', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 22086}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6814', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 22086}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6813', 'nonce': 22086}]}, 'public_addr': '172.21.15.33:6809/22086', 'cluster_addr': '172.21.15.33:6811/22086', 'heartbeat_back_addr': '172.21.15.33:6815/22086', 'heartbeat_front_addr': '172.21.15.33:6813/22086', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fcc939b7-c3fb-4117-af41-9eec226f3020', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6817', 'nonce': 22088}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6818', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 22088}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 22088}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 22088}]}, 'public_addr': '172.21.15.33:6817/22088', 'cluster_addr': '172.21.15.33:6819/22088', 'heartbeat_back_addr': '172.21.15.33:6823/22088', 'heartbeat_front_addr': '172.21.15.33:6821/22088', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57f1d668-f85b-4b0c-9093-fb5fef4b91ef', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6824', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6825', 'nonce': 22089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6826', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6827', 'nonce': 22089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6830', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6831', 'nonce': 22089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6828', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6829', 'nonce': 22089}]}, 'public_addr': '172.21.15.33:6825/22089', 'cluster_addr': '172.21.15.33:6827/22089', 'heartbeat_back_addr': '172.21.15.33:6831/22089', 'heartbeat_front_addr': '172.21.15.33:6829/22089', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a274d12c-8683-491d-8656-5af5d12164ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6800', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6801', 'nonce': 22096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6802', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6803', 'nonce': 22096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6806', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6807', 'nonce': 22096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6804', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6805', 'nonce': 22096}]}, 'public_addr': '172.21.15.47:6801/22096', 'cluster_addr': '172.21.15.47:6803/22096', 'heartbeat_back_addr': '172.21.15.47:6807/22096', 'heartbeat_front_addr': '172.21.15.47:6805/22096', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f0a90a47-9b16-46f6-a2a7-afa4a6f2a298', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6816', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6817', 'nonce': 22097}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6818', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6819', 'nonce': 22097}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6822', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6823', 'nonce': 22097}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6820', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6821', 'nonce': 22097}]}, 'public_addr': '172.21.15.47:6817/22097', 'cluster_addr': '172.21.15.47:6819/22097', 'heartbeat_back_addr': '172.21.15.47:6823/22097', 'heartbeat_front_addr': '172.21.15.47:6821/22097', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '194e23d1-7f52-43ee-8557-52d75a3c22dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6808', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6809', 'nonce': 22108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6810', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6811', 'nonce': 22108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6814', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6815', 'nonce': 22108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6812', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6813', 'nonce': 22108}]}, 'public_addr': '172.21.15.47:6809/22108', 'cluster_addr': '172.21.15.47:6811/22108', 'heartbeat_back_addr': '172.21.15.47:6815/22108', 'heartbeat_front_addr': '172.21.15.47:6813/22108', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '1d1c61e7-be03-4c72-bc95-e187eb95a1b8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6824', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6825', 'nonce': 22109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6826', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6827', 'nonce': 22109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6830', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6831', 'nonce': 22109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6828', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6829', 'nonce': 22109}]}, 'public_addr': '172.21.15.47:6825/22109', 'cluster_addr': '172.21.15.47:6827/22109', 'heartbeat_back_addr': '172.21.15.47:6831/22109', 'heartbeat_front_addr': '172.21.15.47:6829/22109', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '1f23463d-2214-416b-b3ba-23132f2d84ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6801', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6803', 'nonce': 22357}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6805', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6807', 'nonce': 22357}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6813', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6815', 'nonce': 22357}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6809', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6811', 'nonce': 22357}]}, 'public_addr': '172.21.15.155:6803/22357', 'cluster_addr': '172.21.15.155:6807/22357', 'heartbeat_back_addr': '172.21.15.155:6815/22357', 'heartbeat_front_addr': '172.21.15.155:6811/22357', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '39226fc7-a654-41c6-9d70-5ea2f69c236b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6800', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6802', 'nonce': 22359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6804', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6806', 'nonce': 22359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6812', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6814', 'nonce': 22359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6808', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6810', 'nonce': 22359}]}, 'public_addr': '172.21.15.155:6802/22359', 'cluster_addr': '172.21.15.155:6806/22359', 'heartbeat_back_addr': '172.21.15.155:6814/22359', 'heartbeat_front_addr': '172.21.15.155:6810/22359', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'eaafc17e-407d-43aa-8083-2c56660d7db4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6816', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6817', 'nonce': 22358}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6818', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6819', 'nonce': 22358}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6822', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6823', 'nonce': 22358}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6820', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6821', 'nonce': 22358}]}, 'public_addr': '172.21.15.155:6817/22358', 'cluster_addr': '172.21.15.155:6819/22358', 'heartbeat_back_addr': '172.21.15.155:6823/22358', 'heartbeat_front_addr': '172.21.15.155:6821/22358', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '05fdd01d-8c78-4a61-81e7-1d9816e951a3', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6824', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6825', 'nonce': 22360}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6826', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6827', 'nonce': 22360}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6830', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6831', 'nonce': 22360}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6828', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6829', 'nonce': 22360}]}, 'public_addr': '172.21.15.155:6825/22360', 'cluster_addr': '172.21.15.155:6827/22360', 'heartbeat_back_addr': '172.21.15.155:6831/22360', 'heartbeat_front_addr': '172.21.15.155:6829/22360', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.101167+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.166075+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.012320+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.227835+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.010828+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.170791+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.037098+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.281668+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:49.661197+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.009199+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.143021+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.187329+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [11, 9, 5, 4, 6, 10, 0, 1]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'lrcprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'l': '3', 'm': '2', 'name': 'lrcprofile', 'plugin': 'lrc'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 19, 'length': 1}, {'begin': 21, 'length': 1}, {'begin': 24, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T01:44:40.989 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:44:40.990 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:40.990 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:40.990 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:40.990 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.454006853999999, 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.451575456, 2024-09-15T01:44:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.451336641000001, 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.451305258000001, 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.472097827999999, 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.464183426000002, 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.464154163, 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.436717316999999, 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.435550572, 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.435177298999999, 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.452243631000002, 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.464578843999998, 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.462340383000001, 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.461770714, 2024-09-15T01:44:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.461711745999999, 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=13, reqid=client.4632.0:302, at_version=43'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.460118867999999, 2024-09-15T01:44:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094416429999999996, 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520336+0000", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520377+0000", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1192000000000001e-05 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520398+0000", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1013e-05 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520411+0000", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.239e-05 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522362+0000", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001951803 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522402+0000", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9764000000000001e-05 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529721+0000", 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007318504 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529777+0000", 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6501e-05 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.473813538, 2024-09-15T01:44:41.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.47371976, 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:41.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.407075646, 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:44:41.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:44:41.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:494 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~606208] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.4024755310000003, 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010514348, 2024-09-15T01:44:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 494 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577981+0000", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0770000000000002e-06 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578273+0000", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000291797 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578292+0000", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9491e-05 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578300+0000", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5530000000000004e-06 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578395+0000", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4924000000000001e-05 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578567+0000", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172413 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.580077+0000", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001509828 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.581334+0000", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012575889999999999 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588449+0000", 2024-09-15T01:44:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071140400000000003 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588493+0000", 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4635999999999998e-05 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.096 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:41.098 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:41.100 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:41.095+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:41.199 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:41.192+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:41.247 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.247 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:41.247 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:41.247 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:41.300 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:41.291+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:41.357 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.357 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:41.357 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:41.357 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:41.357 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:41.400 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:41.391+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:44:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:44:41.457 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-15T01:44:40.594550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":50840115,"num_objects":46,"num_object_clones":2,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":300,"num_read_kb":48318,"num_write":462,"num_write_kb":63656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":39703091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":511,"ondisk_log_size":511,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":10,"kb":943718400,"kb_used":3082388,"kb_used_data":108792,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":940636012,"statfs":{"total":966367641600,"available":963211276288,"internally_reserved":0,"allocated":111403008,"data_stored":117337220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1572864,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2818,"num_write":24,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.090548"},"pg_stats":[{"pgid":"3.f","version":"55'28","reported_seq":174,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:44:37.571017+0000","last_change":"2024-09-15T01:44:37.571479+0000","last_active":"2024-09-15T01:44:37.571017+0000","last_peered":"2024-09-15T01:44:37.571017+0000","last_clean":"2024-09-15T01:44:37.571017+0000","last_became_active":"2024-09-15T01:44:14.525529+0000","last_became_peered":"2024-09-15T01:44:14.525529+0000","last_unstale":"2024-09-15T01:44:37.571017+0000","last_undegraded":"2024-09-15T01:44:37.571017+0000","last_fullsized":"2024-09-15T01:44:37.571017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057036399999999996,"stat_sum":{"num_bytes":6160384,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2169,"num_write":22,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,1,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.e","version":"40'9","reported_seq":115,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.551627+0000","last_change":"2024-09-15T01:44:34.551627+0000","last_active":"2024-09-15T01:44:34.551627+0000","last_peered":"2024-09-15T01:44:34.551627+0000","last_clean":"2024-09-15T01:44:34.551627+0000","last_became_active":"2024-09-15T01:44:14.525437+0000","last_became_peered":"2024-09-15T01:44:14.525437+0000","last_unstale":"2024-09-15T01:44:34.551627+0000","last_undegraded":"2024-09-15T01:44:34.551627+0000","last_fullsized":"2024-09-15T01:44:34.551627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.758952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088927899999999998,"stat_sum":{"num_bytes":475136,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":6,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.d","version":"55'10","reported_seq":190,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T01:44:37.570937+0000","last_change":"2024-09-15T01:44:37.571830+0000","last_active":"2024-09-15T01:44:37.570937+0000","last_peered":"2024-09-15T01:44:37.570937+0000","last_clean":"2024-09-15T01:44:37.570937+0000","last_became_active":"2024-09-15T01:44:33.552142+0000","last_became_peered":"2024-09-15T01:44:33.552142+0000","last_unstale":"2024-09-15T01:44:37.570937+0000","last_undegraded":"2024-09-15T01:44:37.570937+0000","last_fullsized":"2024-09-15T01:44:37.570937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.473466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001067704,"stat_sum":{"num_bytes":1540096,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2819,"num_write":26,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3080192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,9,6,1,3,4,0],"acting":[7,10,9,6,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"56'41","reported_seq":180,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:44:35.601252+0000","last_change":"2024-09-15T01:44:34.552521+0000","last_active":"2024-09-15T01:44:35.601252+0000","last_peered":"2024-09-15T01:44:35.601252+0000","last_clean":"2024-09-15T01:44:35.601252+0000","last_became_active":"2024-09-15T01:44:33.554020+0000","last_became_peered":"2024-09-15T01:44:33.554020+0000","last_unstale":"2024-09-15T01:44:35.601252+0000","last_undegraded":"2024-09-15T01:44:35.601252+0000","last_fullsized":"2024-09-15T01:44:35.601252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:15.891059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011625229999999999,"stat_sum":{"num_bytes":5718016,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":23,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,10,9,2,0,7],"acting":[4,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.b","version":"40'25","reported_seq":152,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.561404+0000","last_change":"2024-09-15T01:44:34.561404+0000","last_active":"2024-09-15T01:44:34.561404+0000","last_peered":"2024-09-15T01:44:34.561404+0000","last_clean":"2024-09-15T01:44:34.561404+0000","last_became_active":"2024-09-15T01:44:14.534602+0000","last_became_peered":"2024-09-15T01:44:14.534602+0000","last_unstale":"2024-09-15T01:44:34.561404+0000","last_undegraded":"2024-09-15T01:44:34.561404+0000","last_fullsized":"2024-09-15T01:44:34.561404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:17.646194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027713049999999999,"stat_sum":{"num_bytes":2637824,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":30,"num_write_kb":2576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.a","version":"46'10","reported_seq":107,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.559301+0000","last_change":"2024-09-15T01:44:34.559301+0000","last_active":"2024-09-15T01:44:34.559301+0000","last_peered":"2024-09-15T01:44:34.559301+0000","last_clean":"2024-09-15T01:44:34.559301+0000","last_became_active":"2024-09-15T01:44:07.579889+0000","last_became_peered":"2024-09-15T01:44:07.579889+0000","last_unstale":"2024-09-15T01:44:34.559301+0000","last_undegraded":"2024-09-15T01:44:34.559301+0000","last_fullsized":"2024-09-15T01:44:34.559301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.248888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057806600000000002,"stat_sum":{"num_bytes":491520,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":7,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.9","version":"55'14","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.575374+0000","last_change":"2024-09-15T01:44:34.575374+0000","last_active":"2024-09-15T01:44:34.575374+0000","last_peered":"2024-09-15T01:44:34.575374+0000","last_clean":"2024-09-15T01:44:34.575374+0000","last_became_active":"2024-09-15T01:44:33.554096+0000","last_became_peered":"2024-09-15T01:44:33.554096+0000","last_unstale":"2024-09-15T01:44:34.575374+0000","last_undegraded":"2024-09-15T01:44:34.575374+0000","last_fullsized":"2024-09-15T01:44:34.575374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:22.584117+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016703121000000001,"stat_sum":{"num_bytes":475136,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,0,1,4,10],"acting":[8,3,7,9,0,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.8","version":"46'18","reported_seq":163,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.554085+0000","last_change":"2024-09-15T01:44:34.554085+0000","last_active":"2024-09-15T01:44:34.554085+0000","last_peered":"2024-09-15T01:44:34.554085+0000","last_clean":"2024-09-15T01:44:34.554085+0000","last_became_active":"2024-09-15T01:44:33.555455+0000","last_became_peered":"2024-09-15T01:44:33.555455+0000","last_unstale":"2024-09-15T01:44:34.554085+0000","last_undegraded":"2024-09-15T01:44:34.554085+0000","last_fullsized":"2024-09-15T01:44:34.554085+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:12.770472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001967079,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":2547,"num_write":8,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,9,10,7],"acting":[8,1,2,4,3,9,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":183,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:44:36.528878+0000","last_change":"2024-09-15T01:44:14.520601+0000","last_active":"2024-09-15T01:44:36.528878+0000","last_peered":"2024-09-15T01:44:36.528878+0000","last_clean":"2024-09-15T01:44:36.528878+0000","last_became_active":"2024-09-15T01:44:14.520194+0000","last_became_peered":"2024-09-15T01:44:14.520194+0000","last_unstale":"2024-09-15T01:44:36.528878+0000","last_undegraded":"2024-09-15T01:44:36.528878+0000","last_fullsized":"2024-09-15T01:44:36.528878+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:43:52.350040+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:59.848114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":25,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.550743+0000","last_change":"2024-09-15T01:44:30.535121+0000","last_active":"2024-09-15T01:44:34.550743+0000","last_peered":"2024-09-15T01:44:34.550743+0000","last_clean":"2024-09-15T01:44:34.550743+0000","last_became_active":"2024-09-15T01:44:30.530642+0000","last_became_peered":"2024-09-15T01:44:30.530642+0000","last_unstale":"2024-09-15T01:44:34.550743+0000","last_undegraded":"2024-09-15T01:44:34.550743+0000","last_fullsized":"2024-09-15T01:44:34.550743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"46'20","reported_seq":160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.605320+0000","last_change":"2024-09-15T01:44:34.564065+0000","last_active":"2024-09-15T01:44:34.605320+0000","last_peered":"2024-09-15T01:44:34.605320+0000","last_clean":"2024-09-15T01:44:34.605320+0000","last_became_active":"2024-09-15T01:44:33.553924+0000","last_became_peered":"2024-09-15T01:44:33.553924+0000","last_unstale":"2024-09-15T01:44:34.605320+0000","last_undegraded":"2024-09-15T01:44:34.605320+0000","last_fullsized":"2024-09-15T01:44:34.605320+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:10.985856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053274059999999998,"stat_sum":{"num_bytes":1523712,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":5430,"num_write":12,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,8,1,2,10],"acting":[3,9,6,4,8,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T01:44:36.528682+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:44:36.528682+0000","last_peered":"2024-09-15T01:44:36.528682+0000","last_clean":"2024-09-15T01:44:36.528682+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:44:36.528682+0000","last_undegraded":"2024-09-15T01:44:36.528682+0000","last_fullsized":"2024-09-15T01:44:36.528682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"55'38","reported_seq":226,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.684056+0000","last_change":"2024-09-15T01:44:34.578278+0000","last_active":"2024-09-15T01:44:34.684056+0000","last_peered":"2024-09-15T01:44:34.684056+0000","last_clean":"2024-09-15T01:44:34.684056+0000","last_became_active":"2024-09-15T01:44:33.551949+0000","last_became_peered":"2024-09-15T01:44:33.551949+0000","last_unstale":"2024-09-15T01:44:34.684056+0000","last_undegraded":"2024-09-15T01:44:34.684056+0000","last_fullsized":"2024-09-15T01:44:34.684056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.694050+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.019727626000000002,"stat_sum":{"num_bytes":6782976,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3657,"num_write":24,"num_write_kb":6800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8044544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,8,9,6,1],"acting":[0,4,10,3,8,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.550526+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:44:34.550526+0000","last_peered":"2024-09-15T01:44:34.550526+0000","last_clean":"2024-09-15T01:44:34.550526+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:44:34.550526+0000","last_undegraded":"2024-09-15T01:44:34.550526+0000","last_fullsized":"2024-09-15T01:44:34.550526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:03.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"55'27","reported_seq":151,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.565235+0000","last_change":"2024-09-15T01:44:34.565235+0000","last_active":"2024-09-15T01:44:34.565235+0000","last_peered":"2024-09-15T01:44:34.565235+0000","last_clean":"2024-09-15T01:44:34.565235+0000","last_became_active":"2024-09-15T01:44:33.548478+0000","last_became_peered":"2024-09-15T01:44:33.548478+0000","last_unstale":"2024-09-15T01:44:34.565235+0000","last_undegraded":"2024-09-15T01:44:34.565235+0000","last_fullsized":"2024-09-15T01:44:34.565235+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.310629+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014233988,"stat_sum":{"num_bytes":3145728,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":3705,"num_write":18,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,6,1,7,8,0,3],"acting":[4,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":41,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.552559+0000","last_change":"2024-09-15T01:44:33.542100+0000","last_active":"2024-09-15T01:44:34.552559+0000","last_peered":"2024-09-15T01:44:34.552559+0000","last_clean":"2024-09-15T01:44:34.552559+0000","last_became_active":"2024-09-15T01:44:33.541651+0000","last_became_peered":"2024-09-15T01:44:33.541651+0000","last_unstale":"2024-09-15T01:44:34.552559+0000","last_undegraded":"2024-09-15T01:44:34.552559+0000","last_fullsized":"2024-09-15T01:44:34.552559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:18.093385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"46'15","reported_seq":182,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.554165+0000","last_change":"2024-09-15T01:44:34.554165+0000","last_active":"2024-09-15T01:44:34.554165+0000","last_peered":"2024-09-15T01:44:34.554165+0000","last_clean":"2024-09-15T01:44:34.554165+0000","last_became_active":"2024-09-15T01:44:33.554140+0000","last_became_peered":"2024-09-15T01:44:33.554140+0000","last_unstale":"2024-09-15T01:44:34.554165+0000","last_undegraded":"2024-09-15T01:44:34.554165+0000","last_fullsized":"2024-09-15T01:44:34.554165+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.139848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001176372,"stat_sum":{"num_bytes":1622016,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1347,"num_write":9,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2277376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,8,9,10],"acting":[4,3,0,6,2,8,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"3.5","version":"56'75","reported_seq":236,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:44:35.598808+0000","last_change":"2024-09-15T01:44:34.578169+0000","last_active":"2024-09-15T01:44:35.598808+0000","last_peered":"2024-09-15T01:44:35.598808+0000","last_clean":"2024-09-15T01:44:35.598808+0000","last_became_active":"2024-09-15T01:44:33.551490+0000","last_became_peered":"2024-09-15T01:44:33.551490+0000","last_unstale":"2024-09-15T01:44:35.598808+0000","last_undegraded":"2024-09-15T01:44:35.598808+0000","last_fullsized":"2024-09-15T01:44:35.598808+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:30.747962+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.019135587999999999,"stat_sum":{"num_bytes":7815168,"num_objects":8,"num_object_clones":1,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":26,"num_read_kb":2317,"num_write":45,"num_write_kb":10384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5160960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,6,7,1,8,10],"acting":[9,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.553134+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:44:34.553134+0000","last_peered":"2024-09-15T01:44:34.553134+0000","last_clean":"2024-09-15T01:44:34.553134+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:44:34.553134+0000","last_undegraded":"2024-09-15T01:44:34.553134+0000","last_fullsized":"2024-09-15T01:44:34.553134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"55'41","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:44:35.609329+0000","last_change":"2024-09-15T01:44:34.568801+0000","last_active":"2024-09-15T01:44:35.609329+0000","last_peered":"2024-09-15T01:44:35.609329+0000","last_clean":"2024-09-15T01:44:35.609329+0000","last_became_active":"2024-09-15T01:44:33.548174+0000","last_became_peered":"2024-09-15T01:44:33.548174+0000","last_unstale":"2024-09-15T01:44:35.609329+0000","last_undegraded":"2024-09-15T01:44:35.609329+0000","last_fullsized":"2024-09-15T01:44:35.609329+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:11.395908+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016019142,"stat_sum":{"num_bytes":6176768,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":18137,"num_write":49,"num_write_kb":6304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9895936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,3,10,2,6,0],"acting":[1,8,9,3,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.558303+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:44:34.558303+0000","last_peered":"2024-09-15T01:44:34.558303+0000","last_clean":"2024-09-15T01:44:34.558303+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:44:34.558303+0000","last_undegraded":"2024-09-15T01:44:34.558303+0000","last_fullsized":"2024-09-15T01:44:34.558303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"55'20","reported_seq":150,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.578958+0000","last_change":"2024-09-15T01:44:34.578958+0000","last_active":"2024-09-15T01:44:34.578958+0000","last_peered":"2024-09-15T01:44:34.578958+0000","last_clean":"2024-09-15T01:44:34.578958+0000","last_became_active":"2024-09-15T01:44:14.534165+0000","last_became_peered":"2024-09-15T01:44:14.534165+0000","last_unstale":"2024-09-15T01:44:34.578958+0000","last_undegraded":"2024-09-15T01:44:34.578958+0000","last_fullsized":"2024-09-15T01:44:34.578958+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.075163+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.019593877999999999,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1255,"num_write":33,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.552868+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:44:34.552868+0000","last_peered":"2024-09-15T01:44:34.552868+0000","last_clean":"2024-09-15T01:44:34.552868+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:44:34.552868+0000","last_undegraded":"2024-09-15T01:44:34.552868+0000","last_fullsized":"2024-09-15T01:44:34.552868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"55'25","reported_seq":163,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T01:44:35.598573+0000","last_change":"2024-09-15T01:44:34.559621+0000","last_active":"2024-09-15T01:44:35.598573+0000","last_peered":"2024-09-15T01:44:35.598573+0000","last_clean":"2024-09-15T01:44:35.598573+0000","last_became_active":"2024-09-15T01:44:33.551249+0000","last_became_peered":"2024-09-15T01:44:33.551249+0000","last_unstale":"2024-09-15T01:44:35.598573+0000","last_undegraded":"2024-09-15T01:44:35.598573+0000","last_fullsized":"2024-09-15T01:44:35.598573+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.245186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052012599999999999,"stat_sum":{"num_bytes":3227648,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4567,"num_write":19,"num_write_kb":4576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,6],"acting":[0,1,4,7,8,10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"2"},{"start":"15","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T01:44:34.558751+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:44:34.558751+0000","last_peered":"2024-09-15T01:44:34.558751+0000","last_clean":"2024-09-15T01:44:34.558751+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:44:34.558751+0000","last_undegraded":"2024-09-15T01:44:34.558751+0000","last_fullsized":"2024-09-15T01:44:34.558751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":50249728,"num_objects":42,"num_object_clones":2,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":224,"num_read_kb":48254,"num_write":337,"num_write_kb":61504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":39174144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118140928,"data_stored":128278528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":416,"ondisk_log_size":416,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310668,"kb_used_data":13308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94061172,"statfs":{"total":96636764160,"available":96318640128,"internally_reserved":0,"allocated":13627392,"data_stored":15820052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444043,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444044,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308852,"kb_used_data":11492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062988,"statfs":{"total":96636764160,"available":96320499712,"internally_reserved":0,"allocated":11767808,"data_stored":12472592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":9,"up_from":17,"seq":73014444043,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310264,"kb_used_data":12904,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94061576,"statfs":{"total":96636764160,"available":96319053824,"internally_reserved":0,"allocated":13213696,"data_stored":13918480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444043,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308028,"kb_used_data":10668,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063812,"statfs":{"total":96636764160,"available":96321343488,"internally_reserved":0,"allocated":10924032,"data_stored":11006224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444044,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308452,"kb_used_data":11092,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063388,"statfs":{"total":96636764160,"available":96320909312,"internally_reserved":0,"allocated":11358208,"data_stored":12419907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]},{"osd":5,"up_from":0,"seq":73014444044,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":17,"seq":73014444043,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306848,"kb_used_data":9488,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064992,"statfs":{"total":96636764160,"available":96322551808,"internally_reserved":0,"allocated":9715712,"data_stored":10055952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444043,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307564,"kb_used_data":10208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064276,"statfs":{"total":96636764160,"available":96321818624,"internally_reserved":0,"allocated":10452992,"data_stored":10174558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.675}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.95}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7270000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7909999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444044,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305796,"kb_used_data":8436,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066044,"statfs":{"total":96636764160,"available":96323629056,"internally_reserved":0,"allocated":8638464,"data_stored":9343248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444043,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308476,"kb_used_data":11116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063364,"statfs":{"total":96636764160,"available":96320884736,"internally_reserved":0,"allocated":11382784,"data_stored":11460899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9179999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9670000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444043,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307440,"kb_used_data":10080,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064400,"statfs":{"total":96636764160,"available":96321945600,"internally_reserved":0,"allocated":10321920,"data_stored":10665308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9965568,"data_stored":10588160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10727424,"data_stored":11091968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7987200,"data_stored":8974336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10215424,"data_stored":11202560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9064448,"data_stored":9687040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8859648,"data_stored":9601024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10170368,"data_stored":11157504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12419072,"data_stored":14905344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10272768,"data_stored":10637312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":12562432,"data_stored":13549568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":11116544,"data_stored":12103680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:44:41.459 INFO:tasks.thrashosds.thrasher:Setting 3.a to [11, 0, 4, 9, 7, 3, 6, 2, 8, 5, 10, 1] 2024-09-15T01:44:41.459 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '11', '0', '4', '9', '7', '3', '6', '2', '8', '5', '10', '1'] 2024-09-15T01:44:41.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.a 11 0 4 9 7 3 6 2 8 5 10 1 2024-09-15T01:44:41.470 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.944526007, 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.944487001999999, 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.944456058, 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.949571758000001, 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.947576237, 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:41.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.947506704999999, 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.496 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.502 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.947484168999999, 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:41.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:41.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.947416977, 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:41.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:41.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.947347451999999, 2024-09-15T01:44:41.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.947105374, 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:41.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:41.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.947051940000001, 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.946979761000001, 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:41.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.933018203, 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.537 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:41.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:498 3.4s0 3:25aaa82b:::smithi03327189-9:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8614672690000003, 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011653834, 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 498 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604049+0000", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8739999999999997e-06 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604468+0000", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041931899999999997 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604508+0000", 2024-09-15T01:44:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9677e-05 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604524+0000", 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6030000000000001e-05 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604637+0000", 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113602 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604816+0000", 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606494+0000", 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016776149999999999 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615523+0000", 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0090292470000000007 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615637+0000", 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011338200000000001 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615699+0000", 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1970999999999993e-05 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:502 3.4s0 3:25aaa82b:::smithi03327189-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.548 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8510394489999999, 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010446636, 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 502 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614473+0000", 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614474+0000", 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0020000000000001e-06 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614481+0000", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9920000000000001e-06 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614501+0000", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0412e-05 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.614509+0000", 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9920000000000007e-06 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619531+0000", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050217589999999998 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.619623+0000", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2584000000000007e-05 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.623447+0000", 2024-09-15T01:44:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038235180000000001 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.623538+0000", 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1099000000000003e-05 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.624919+0000", 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013812780000000001 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:509 3.4s0 3:20577364:::smithi03327189-24:head [append 0~32768] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.635953+0000", 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.829559572, 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010094053, 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 509 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635953+0000", 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635953+0000", 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635954+0000", 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.085e-06 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635971+0000", 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462999999999999e-05 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635979+0000", 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7600000000000002e-06 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635984+0000", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0220000000000004e-06 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.637767+0000", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001782861 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.637878+0000", 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111383 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.638736+0000", 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00085809099999999995 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.641576+0000", 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028400539999999998 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645972+0000", 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043962259999999996 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646047+0000", 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4108000000000003e-05 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:510 3.4s0 3:20577364:::smithi03327189-24:head [setxattr _header (57) in=7b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.635993+0000", 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8295194930000003, 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011117019000000001, 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 510 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635993+0000", 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635993+0000", 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635993+0000", 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5000000000000003e-07 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.635996+0000", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1729999999999999e-06 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636005+0000", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7870000000000007e-06 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636009+0000", 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7799999999999998e-06 2024-09-15T01:44:41.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639668+0000", 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036588469999999998 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.639791+0000", 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122798 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640178+0000", 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038728200000000002 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.641777+0000", 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001598875 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647013+0000", 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052359449999999997 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647110+0000", 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6982000000000001e-05 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:511 3.4s0 3:20577364:::smithi03327189-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8294997300000002, 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014490421999999999, 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:41.565 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 511 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636013+0000", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636015+0000", 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2869999999999999e-06 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636020+0000", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.34e-06 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636022+0000", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3779999999999999e-06 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640680+0000", 2024-09-15T01:44:41.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046585400000000001 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640820+0000", 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014002599999999999 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647530+0000", 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067096409999999997 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647618+0000", 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7530999999999999e-05 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.650503+0000", 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028851409999999999 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 58/53 ECSubWrite(tid=39, reqid=osd.4.0:39, at_version=58'31, trim_to=0'0, roll_forward_to=57'29))", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.563327+0000", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9021849500000001, 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097561550000000007, 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563327+0000", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563327+0000", 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563328+0000", 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6599999999999995e-07 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563359+0000", 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1482000000000001e-05 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563374+0000", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4205999999999999e-05 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563383+0000", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4970000000000007e-06 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563474+0000", 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1240000000000001e-05 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.563502+0000", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7186999999999999e-05 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.573055+0000", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095532069999999993 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.573083+0000", 2024-09-15T01:44:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8569999999999999e-05 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.6s1 58/53 ECSubWrite(tid=126, reqid=osd.0.0:126, at_version=58'32, trim_to=0'0, roll_forward_to=57'30))", 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.8966011859999998, 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01167437, 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000002e-07 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568942+0000", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1117e-05 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568954+0000", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1612e-05 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568966+0000", 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2051e-05 2024-09-15T01:44:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569058+0000", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2028999999999996e-05 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569071+0000", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2559e-05 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580549+0000", 2024-09-15T01:44:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011478647999999999 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580585+0000", 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6050000000000002e-05 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.579 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:41.582 INFO:tasks.rados.rados.0.smithi033.stdout:239: done (0 left) 2024-09-15T01:44:41.582 INFO:tasks.rados.rados.0.smithi033.stdout:240: read oid 32 snap -1 2024-09-15T01:44:41.582 INFO:tasks.rados.rados.0.smithi033.stdout:240: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:41.582 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 19 current snap is 15 2024-09-15T01:44:41.582 INFO:tasks.rados.rados.0.smithi033.stdout:241: seq_num 119 ranges {1490944=458752,1949696=507904,2457600=507904,2965504=458752,3424256=81920} 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:241: writing smithi03327189-19 from 1490944 to 1949696 tid 1 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:241: writing smithi03327189-19 from 1949696 to 2457600 tid 2 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:241: writing smithi03327189-19 from 2457600 to 2965504 tid 3 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:241: writing smithi03327189-19 from 2965504 to 3424256 tid 4 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:241: writing smithi03327189-19 from 3424256 to 3506176 tid 5 2024-09-15T01:44:41.583 INFO:tasks.rados.rados.0.smithi033.stdout:240: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:41.583 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:41.575+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 1 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 2 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 3 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 4 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 5 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:240: done (1 left) 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:242: snap_create 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 6 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: finishing write tid 7 to smithi03327189-19 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 19 v 26 (ObjNum 119 snap 15 seq_num 119) dirty exists 2024-09-15T01:44:41.605 INFO:tasks.rados.rados.0.smithi033.stdout:241: left oid 19 (ObjNum 119 snap 15 seq_num 119) 2024-09-15T01:44:41.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:41.676+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:41.691 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:41.692 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:41.783 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:41.779+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:41.844 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:41.844 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:41.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:41.883 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:41.880+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.430213096999999, 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 38/34 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=38'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.430062168999999, 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055670379999999998, 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519408+0000", 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519409+0000", 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7599999999999996e-07 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519454+0000", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4817000000000001e-05 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519475+0000", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1217e-05 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519488+0000", 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.26e-05 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520876+0000", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001388877 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520897+0000", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.02e-05 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524944+0000", 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040469709999999999 2024-09-15T01:44:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.524975+0000", 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1479999999999997e-05 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.427727572999999, 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.8s2 39/34 ECSubWrite(tid=34, reqid=client.4632.0:252, at_version=39'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.425204277999999, 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056439819999999996, 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524266+0000", 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.27e-07 2024-09-15T01:44:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524309+0000", 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2670000000000003e-05 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524323+0000", 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3813e-05 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524331+0000", 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2300000000000008e-06 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524482+0000", 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015112700000000001 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524497+0000", 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4666e-05 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529866+0000", 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053691579999999997 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.529910+0000", 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3990999999999999e-05 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 43/34 ECSubWrite(tid=135, reqid=client.4632.0:304, at_version=43'12, trim_to=0'0, roll_forward_to=40'10))", 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.400436075000002, 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059241329999999998, 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549034+0000", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549410+0000", 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000375464 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549431+0000", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713999999999999e-05 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549438+0000", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549499+0000", 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1303999999999994e-05 2024-09-15T01:44:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549514+0000", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4901e-05 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554929+0000", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054145510000000001 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.554958+0000", 2024-09-15T01:44:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9393000000000001e-05 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4202243210000001, 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:44:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4170560300000004, 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063209149999999999, 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532415+0000", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534495+0000", 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020793859999999999 2024-09-15T01:44:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534525+0000", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0420999999999999e-05 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534538+0000", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3022e-05 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534680+0000", 2024-09-15T01:44:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014239299999999999 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534711+0000", 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0543999999999998e-05 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538705+0000", 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039944020000000002 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538735+0000", 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9815000000000002e-05 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:515 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~557056] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.3093307459999997, 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058152500000000001, 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 515 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640139+0000", 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640141+0000", 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.829e-06 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640648+0000", 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050664500000000001 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640663+0000", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5172e-05 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640670+0000", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2749999999999998e-06 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640726+0000", 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5825000000000002e-05 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640828+0000", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101492 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.642266+0000", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014377940000000001 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645206+0000", 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002940645 2024-09-15T01:44:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645866+0000", 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066034100000000005 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645955+0000", 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8232000000000005e-05 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.30409256, 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:44:41.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:41.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:41.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:517 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.3000199910000001, 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062744919999999996, 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 517 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649450+0000", 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649452+0000", 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2350000000000002e-06 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649713+0000", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026092800000000001 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649729+0000", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5889999999999999e-05 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649740+0000", 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0368e-05 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649784+0000", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4493999999999998e-05 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649889+0000", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105344 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.651145+0000", 2024-09-15T01:44:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001255649 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655608+0000", 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044630939999999999 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655660+0000", 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1365e-05 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655725+0000", 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5124999999999997e-05 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:518 3.es0 3:7de15fb4:::smithi03327189-49:head [setxattr _header (57) in=7b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.2996935350000003, 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063479770000000003, 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 518 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:42.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649778+0000", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.144e-06 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649784+0000", 2024-09-15T01:44:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8270000000000003e-06 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649794+0000", 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0713999999999999e-05 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649800+0000", 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2429999999999996e-06 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652010+0000", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002210231 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652118+0000", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108608 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652627+0000", 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050885000000000004 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655730+0000", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031025250000000001 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655776+0000", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5812000000000001e-05 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656125+0000", 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034902300000000001 2024-09-15T01:44:42.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.2996639500000002, 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:44:42.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:44:42.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:548 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645189+0000", 2024-09-15T01:44:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3042811639999998, 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059188799999999996, 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 548 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645189+0000", 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645189+0000", 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645191+0000", 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7069999999999999e-06 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645204+0000", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2829e-05 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645226+0000", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2310000000000002e-05 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645233+0000", 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.075e-06 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645314+0000", 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1522999999999999e-05 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645443+0000", 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012879999999999999 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.651108+0000", 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056646359999999998 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:549 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3042143590000004, 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072147799999999996, 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 549 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3900000000000005e-07 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645259+0000", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8839999999999998e-06 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645268+0000", 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4149999999999999e-06 2024-09-15T01:44:42.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645271+0000", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.841e-06 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645937+0000", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066678699999999996 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.646068+0000", 2024-09-15T01:44:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013108799999999999 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.652471+0000", 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064022259999999996 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3041719010000001, 2024-09-15T01:44:42.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:44:42.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:44:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1986998250000003, 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:44:42.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1952602509999997, 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:44:42.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:565 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~655360] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1909142609999996, 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069304919999999999, 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 565 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.043 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758558+0000", 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9139999999999998e-06 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758778+0000", 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000220263 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758805+0000", 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7084e-05 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758816+0000", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1148999999999999e-05 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759223+0000", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040634800000000001 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759333+0000", 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110512 2024-09-15T01:44:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.760874+0000", 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001540689 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764491+0000", 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036169409999999998 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765445+0000", 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00095386899999999998 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765486+0000", 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1723e-05 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:566 3.es0 3:720ccbad:::smithi03327189-29:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1906393810000004, 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071089170000000002, 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 566 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758832+0000", 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2500000000000001e-06 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758838+0000", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6380000000000001e-06 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758856+0000", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8034999999999999e-05 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758955+0000", 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8777000000000007e-05 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.762040+0000", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003085951 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.762328+0000", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028736799999999998 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.763596+0000", 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012677560000000001 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764653+0000", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010569170000000001 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765890+0000", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001237036 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765940+0000", 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0188999999999997e-05 2024-09-15T01:44:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1904743099999999, 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:44:42.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.059 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:42.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:42.061 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:42.056+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:42.162 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:42.159+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:42.210 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:42.210 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:42.210 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:42.210 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:42.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:42.263 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:42.256+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:42.322 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:42.413 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:42.413 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:42.413 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:42.413 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:42.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:42.415 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:42.407+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:42.515 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:42.508+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 31.055432326999998, 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.569 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:42.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 31.055378088000001, 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:42.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.577 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 31.055358949999999, 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:42.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/32 ECSubWrite(tid=1, reqid=client.4632.0:219, at_version=33'5, trim_to=0'0, roll_forward_to=0'0, backfill_or_async_recovery))", 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:42.582 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.032655937000001, 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0083125319999999992, 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532175+0000", 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532176+0000", 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1800000000000004e-07 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532200+0000", 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4355999999999999e-05 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532212+0000", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2191000000000001e-05 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532219+0000", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6229999999999999e-06 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532292+0000", 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3135999999999999e-05 2024-09-15T01:44:42.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532304+0000", 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2481e-05 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540422+0000", 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081175480000000005 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.540487+0000", 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5278999999999996e-05 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.04638405, 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:42.592 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.046316388000001, 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:42.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:42.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.046290646999999, 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:42.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.033219838000001, 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.032177933, 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.031357128, 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0344812699999997, 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084277399999999995, 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530350+0000", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530476+0000", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012622100000000001 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530495+0000", 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8326e-05 2024-09-15T01:44:42.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530506+0000", 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1063999999999999e-05 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530816+0000", 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030996700000000002 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530831+0000", 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5129e-05 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538759+0000", 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079275890000000005 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538777+0000", 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0321810300000003, 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089560869999999997, 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532651+0000", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5239999999999999e-06 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532968+0000", 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031664399999999999 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532988+0000", 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0676e-05 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533000+0000", 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1318000000000001e-05 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533071+0000", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0779999999999997e-05 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533082+0000", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1447000000000001e-05 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541581+0000", 2024-09-15T01:44:42.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084990299999999994 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541606+0000", 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4668000000000002e-05 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:481 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~475136] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0293939170000002, 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095211319999999999, 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 481 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:42.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535439+0000", 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5320000000000001e-06 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535767+0000", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032738100000000002 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535789+0000", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1917e-05 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535882+0000", 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3171000000000006e-05 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535944+0000", 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2124000000000004e-05 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.536068+0000", 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012412600000000001 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.537727+0000", 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016587010000000001 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544825+0000", 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070983599999999997 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544893+0000", 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8188999999999994e-05 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544958+0000", 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4630999999999998e-05 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:482 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~442368] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0256829280000002, 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087432670000000007, 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 482 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539150+0000", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1459999999999999e-06 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539423+0000", 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000273195 2024-09-15T01:44:42.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539441+0000", 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7725999999999999e-05 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539449+0000", 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7700000000000001e-06 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539516+0000", 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7158999999999999e-05 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539662+0000", 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000145861 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.540574+0000", 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091271500000000003 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544826+0000", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042521360000000001 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547817+0000", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029910129999999998 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547891+0000", 2024-09-15T01:44:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3546000000000004e-05 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.4s3 58/53 ECSubWrite(tid=112, reqid=osd.1.0:112, at_version=58'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9951705180000001, 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087782980000000004, 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.638 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2300000000000002e-07 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569683+0000", 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2900000000000001e-05 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569693+0000", 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569698+0000", 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2329999999999997e-06 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.571690+0000", 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001991788 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.571705+0000", 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4713e-05 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.578409+0000", 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067040889999999999 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.578438+0000", 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9844000000000001e-05 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9410207230000003, 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:42.643 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:44:42.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:537 3.bs0 3:dc585df2:::smithi03327189-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9409546469999999, 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011082247999999999, 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 537 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.649 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623877+0000", 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623881+0000", 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623895+0000", 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3091000000000001e-05 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623902+0000", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.153e-06 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624357+0000", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045563899999999998 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624496+0000", 2024-09-15T01:44:42.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013903799999999999 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634734+0000", 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01023717 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634849+0000", 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001154 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634958+0000", 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109345 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 58/53 ECSubWrite(tid=36, reqid=client.4632.0:528, at_version=58'91, trim_to=0'0, roll_forward_to=58'89))", 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9368009329999998, 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085050799999999999, 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2399999999999998e-07 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628216+0000", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018574499999999999 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628228+0000", 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1625e-05 2024-09-15T01:44:42.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628234+0000", 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1299999999999998e-06 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628280+0000", 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5904e-05 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628290+0000", 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0180000000000001e-05 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.636504+0000", 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082141460000000003 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.636535+0000", 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 58/53 ECSubWrite(tid=49, reqid=client.4632.0:538, at_version=0'0, trim_to=0'0, roll_forward_to=58'38))", 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.632400+0000", 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9324304009999995, 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082897430000000005, 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632400+0000", 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632400+0000", 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632401+0000", 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3300000000000001e-07 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632413+0000", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1688999999999999e-05 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632428+0000", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5526e-05 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632435+0000", 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5780000000000004e-06 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632788+0000", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00035309700000000002 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.632801+0000", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3074999999999999e-05 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640663+0000", 2024-09-15T01:44:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078623700000000005 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640690+0000", 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6574999999999999e-05 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:555 3.bs0 3:d32e3b1b:::smithi03327189-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.904658081, 2024-09-15T01:44:42.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094173030000000001, 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 555 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660174+0000", 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:42.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660177+0000", 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7069999999999998e-06 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660199+0000", 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1659999999999999e-05 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660205+0000", 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7869999999999998e-06 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663351+0000", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031458609999999998 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663435+0000", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3947000000000001e-05 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668442+0000", 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050072629999999996 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668572+0000", 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013023000000000001 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.669590+0000", 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010177529999999999 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:42.668 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:42.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:44:42.670 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:42.663+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:42.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:42.672 INFO:tasks.rados.rados.0.smithi033.stdout:241: done (1 left) 2024-09-15T01:44:42.672 INFO:tasks.rados.rados.0.smithi033.stdout:242: done (0 left) 2024-09-15T01:44:42.673 INFO:tasks.rados.rados.0.smithi033.stdout:243: copy_from oid 20 from oid 19 current snap is 16 2024-09-15T01:44:42.673 INFO:tasks.rados.rados.0.smithi033.stdout:244: snap_remove snap 13 2024-09-15T01:44:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:set 3.a pg_upmap_items mapping to [11->0,4->9,7->3,6->2,8->5,10->1] 2024-09-15T01:44:42.770 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:42.763+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:42.787 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:42.787 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:42.787 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:42.787 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:42.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:42.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:42.868+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:42.900 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:42.901 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:42.901 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:42.901 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:42.901 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:42.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:42.972 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:42.967+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:43.010 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.486597821, 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:44:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:44:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:44:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.486539475000001, 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:44:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:43.022 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.486460718, 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:43.023 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:44:43.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.486280717, 2024-09-15T01:44:43.028 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:43.029 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.030 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:44:43.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.4759707620000002, 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.034 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.036 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.037 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4207250160000005, 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.040 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:43.041 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:44:43.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.043 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:44:43.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:44:43.045 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:522 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~786432] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:43.046 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4151958929999999, 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078988010000000004, 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 522 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590054+0000", 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590056+0000", 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7269999999999999e-06 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590849+0000", 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00079286600000000004 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590866+0000", 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7288000000000001e-05 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590872+0000", 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7169999999999996e-06 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.590926+0000", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4212999999999997e-05 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.591041+0000", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115254 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592382+0000", 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013405839999999999 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.596905+0000", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045232959999999996 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597832+0000", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000926299 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597953+0000", 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121557 2024-09-15T01:44:43.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:523 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~376832] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4126048689999999, 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080569660000000005, 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 523 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592645+0000", 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.592647+0000", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0269999999999998e-06 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593297+0000", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00064929400000000002 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593319+0000", 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2439e-05 2024-09-15T01:44:43.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593329+0000", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4010000000000003e-06 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593780+0000", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00045140499999999999 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593878+0000", 2024-09-15T01:44:43.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.7739999999999996e-05 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.594895+0000", 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0010174769999999999 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.596948+0000", 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0020530800000000001 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.056 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.057 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.600592+0000", 2024-09-15T01:44:43.057 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036435949999999999 2024-09-15T01:44:43.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.057 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.600702+0000", 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011050800000000001 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4118829469999996, 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:44:43.062 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.063 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3810033260000001, 2024-09-15T01:44:43.064 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:44:43.066 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.067 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.068 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:44:43.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3809239580000003, 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.070 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:44:43.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:43.073 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.074 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:538 3.0s0 3:0899e339:::smithi03327189-38:head [delete] snapc 19=[18] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3808134379999997, 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0080048380000000002, 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.075 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 538 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624437+0000", 2024-09-15T01:44:43.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624438+0000", 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2559999999999999e-06 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624442+0000", 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9099999999999998e-06 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624456+0000", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4392e-05 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624465+0000", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1629999999999996e-06 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625251+0000", 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00078665199999999999 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625564+0000", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000313126 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.626639+0000", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0010743 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.631711+0000", 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050726390000000003 2024-09-15T01:44:43.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.632343+0000", 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00063175199999999996 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.632442+0000", 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8647999999999995e-05 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3610335769999997, 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.081 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:44:43.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.083 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:44:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:542 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~196608] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.359894733, 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0086966509999999997, 2024-09-15T01:44:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 542 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645356+0000", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645357+0000", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.057e-06 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645555+0000", 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000198593 2024-09-15T01:44:43.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645569+0000", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4181e-05 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645574+0000", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.5750000000000002e-06 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.646955+0000", 2024-09-15T01:44:43.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013807369999999999 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.647078+0000", 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012335100000000001 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.647826+0000", 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00074837899999999997 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649457+0000", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001630514 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653983+0000", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0045256189999999998 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654052+0000", 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9645000000000002e-05 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.091 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3596709149999997, 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:44:43.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:44:43.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:43.097 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3596563899999996, 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:43.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:44:43.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:547 3.0s0 3:08c156cd:::smithi03327189-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3596079679999997, 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0091027969999999993, 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 547 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645643+0000", 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.79e-07 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645644+0000", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.699e-06 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645650+0000", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.942e-06 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645653+0000", 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.452e-06 2024-09-15T01:44:43.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653028+0000", 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073755519999999996 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653094+0000", 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5718999999999999e-05 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654745+0000", 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016511539999999999 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3372706289999998, 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:43.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:560 3.0s0 3:0899e339:::smithi03327189-38:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3335381799999997, 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090281700000000003, 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 560 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671713+0000", 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:44:43.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671717+0000", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1749999999999998e-06 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671737+0000", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.948e-05 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671744+0000", 2024-09-15T01:44:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9569999999999996e-06 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676355+0000", 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046115949999999996 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676528+0000", 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017328999999999999 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:43.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677261+0000", 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073246900000000002 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677982+0000", 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072084199999999997 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680660+0000", 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002678541 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680740+0000", 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9963000000000007e-05 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.3335003069999996, 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:43.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:44:43.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:44:43.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:43.123 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:43.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:43.125 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:43.120+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:43.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:43.226 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:43.220+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:43.237 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:43.237 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:43.237 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:43.237 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:43.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:43.285 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:43.306 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:43.327 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:43.320+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:43.327 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:43.342 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:43.343 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:43.343 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:43.350 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:43.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:43.372 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:43.396 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:43.419 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:43.428 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:43.419+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:43.443 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:43.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:43.466 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:43.488 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:43.511 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:43.529 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:43.523+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:43.558 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:43.558 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:43.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:43.613 INFO:tasks.rados.rados.0.smithi033.stdout:244: done (1 left) 2024-09-15T01:44:43.613 INFO:tasks.rados.rados.0.smithi033.stdout:245: setattr oid 30 current snap is 16 2024-09-15T01:44:43.613 INFO:tasks.rados.rados.0.smithi033.stdout:243: finishing copy_from racing read to smithi03327189-20 2024-09-15T01:44:43.614 INFO:tasks.rados.rados.0.smithi033.stdout:243: finishing copy_from to smithi03327189-20 2024-09-15T01:44:43.614 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 26 (ObjNum 119 snap 15 seq_num 119) dirty exists 2024-09-15T01:44:43.615 INFO:tasks.rados.rados.0.smithi033.stdout:243: done (1 left) 2024-09-15T01:44:43.616 INFO:tasks.rados.rados.0.smithi033.stdout:246: read oid 8 snap -1 2024-09-15T01:44:43.616 INFO:tasks.rados.rados.0.smithi033.stdout:246: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-15T01:44:43.616 INFO:tasks.rados.rados.0.smithi033.stdout:247: rollback oid 6 current snap is 16 2024-09-15T01:44:43.616 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 6 to 15 2024-09-15T01:44:43.618 INFO:tasks.rados.rados.0.smithi033.stdout:248: snap_create 2024-09-15T01:44:43.623 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:44:43.630 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 30 v 46 (ObjNum 577 snap 0 seq_num 1713778192) dirty exists 2024-09-15T01:44:43.634 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:43.624+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:43.643 INFO:tasks.rados.rados.0.smithi033.stdout:246: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-15T01:44:43.683 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:43.683 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:43.690 INFO:tasks.rados.rados.0.smithi033.stdout:247: finishing rollback tid 0 to smithi03327189-6 2024-09-15T01:44:43.690 INFO:tasks.rados.rados.0.smithi033.stdout:247: finishing rollback tid 1 to smithi03327189-6 2024-09-15T01:44:43.690 INFO:tasks.rados.rados.0.smithi033.stdout:247: finishing rollback tid 2 to smithi03327189-6 2024-09-15T01:44:43.690 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 66 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:43.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:43.730 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:43.727+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:43.791 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:43.791 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:43.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:43.831 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:43.823+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:43.899 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:43.899 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:43.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:43.932 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:43.928+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:44.010 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.011 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.020 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:44.069 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:44.063+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:44.119 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.119 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.128 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:44.134 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:44.128+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:44.235 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:44.231+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:44.240 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.240 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:44.336 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:44.327+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:44.348 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.348 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:44.436 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:44.432+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:44.453 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.453 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:44.537 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:44.532+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:44.560 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.560 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.569 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:44.631 INFO:tasks.rados.rados.0.smithi033.stdout:245: done (3 left) 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:246: done (2 left) 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:247: done (1 left) 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:248: done (0 left) 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:249: read oid 7 snap -1 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:249: expect (ObjNum 118 snap 14 seq_num 118) 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 45 current snap is 17 2024-09-15T01:44:44.632 INFO:tasks.rados.rados.0.smithi033.stdout:250: seq_num 120 ranges {1622016=475136,2097152=655360,2752512=622592,3375104=278528} 2024-09-15T01:44:44.638 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:44.632+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:44.649 INFO:tasks.rados.rados.0.smithi033.stdout:250: writing smithi03327189-45 from 1622016 to 2097152 tid 1 2024-09-15T01:44:44.653 INFO:tasks.rados.rados.0.smithi033.stdout:250: writing smithi03327189-45 from 2097152 to 2752512 tid 2 2024-09-15T01:44:44.657 INFO:tasks.rados.rados.0.smithi033.stdout:250: writing smithi03327189-45 from 2752512 to 3375104 tid 3 2024-09-15T01:44:44.660 INFO:tasks.rados.rados.0.smithi033.stdout:250: writing smithi03327189-45 from 3375104 to 3653632 tid 4 2024-09-15T01:44:44.660 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 5 current snap is 17 2024-09-15T01:44:44.660 INFO:tasks.rados.rados.0.smithi033.stdout:251: seq_num 121 ranges {81920=491520,573440=802816,1376256=720896,2097152=32768} 2024-09-15T01:44:44.663 INFO:tasks.rados.rados.0.smithi033.stdout:251: writing smithi03327189-5 from 81920 to 573440 tid 1 2024-09-15T01:44:44.668 INFO:tasks.rados.rados.0.smithi033.stdout:251: writing smithi03327189-5 from 573440 to 1376256 tid 2 2024-09-15T01:44:44.673 INFO:tasks.rados.rados.0.smithi033.stdout:251: writing smithi03327189-5 from 1376256 to 2097152 tid 3 2024-09-15T01:44:44.673 INFO:tasks.rados.rados.0.smithi033.stdout:251: writing smithi03327189-5 from 2097152 to 2129920 tid 4 2024-09-15T01:44:44.674 INFO:tasks.rados.rados.0.smithi033.stdout:252: snap_create 2024-09-15T01:44:44.674 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 1 to smithi03327189-45 2024-09-15T01:44:44.674 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 2 to smithi03327189-45 2024-09-15T01:44:44.674 INFO:tasks.rados.rados.0.smithi033.stdout:249: expect (ObjNum 118 snap 14 seq_num 118) 2024-09-15T01:44:44.687 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.687 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.696 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:44.739 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:44.731+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:44.789 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.789 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.800 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:44.822 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 3 to smithi03327189-45 2024-09-15T01:44:44.822 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 4 to smithi03327189-45 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 5 to smithi03327189-45 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 1 to smithi03327189-5 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:250: finishing write tid 6 to smithi03327189-45 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 45 v 104 (ObjNum 120 snap 17 seq_num 120) dirty exists 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:250: left oid 45 (ObjNum 120 snap 17 seq_num 120) 2024-09-15T01:44:44.823 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 2 to smithi03327189-5 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 3 to smithi03327189-5 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 4 to smithi03327189-5 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 5 to smithi03327189-5 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:251: finishing write tid 6 to smithi03327189-5 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 5 v 110 (ObjNum 121 snap 17 seq_num 121) dirty exists 2024-09-15T01:44:44.824 INFO:tasks.rados.rados.0.smithi033.stdout:251: left oid 5 (ObjNum 121 snap 17 seq_num 121) 2024-09-15T01:44:44.839 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:44.831+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:44.899 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:44.899 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:44.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:44.940 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:44.935+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:45.008 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.009 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.017 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:45.041 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:45.036+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:45.117 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.117 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:45.142 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:45.136+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:45.229 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.229 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.238 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:45.243 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:45.236+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:45.335 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.335 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.344 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:45.345 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:45.340+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:45.444 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:45.440+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:45.445 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.445 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:45.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:45.545 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:45.543+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:45.564 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:45.565 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:45.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:45.640 INFO:tasks.rados.rados.0.smithi033.stdout:249: done (3 left) 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:250: done (2 left) 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:251: done (1 left) 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:252: done (0 left) 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:253: copy_from oid 25 from oid 34 current snap is 18 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:254: read oid 3 snap -1 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:254: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:45.641 INFO:tasks.rados.rados.0.smithi033.stdout:255: snap_remove snap 17 2024-09-15T01:44:45.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:44:45.646 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:45.639+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:45.677 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:45.696 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:45.704 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:45.705 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:45.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:45.715 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:45.733 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:45.747 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:45.739+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:45.754 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:45.777 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:45.801 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:45.820 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.290357739999997, 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:45.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.287926341999999, 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.827 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:45.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.287687527000003, 2024-09-15T01:44:45.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:45.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.287656144000003, 2024-09-15T01:44:45.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:45.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.308448714000001, 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:45.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:45.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.851 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.300534312000003, 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:45.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.300505049000002, 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:45.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:45.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.273068203000001, 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:45.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:45.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.271901458000002, 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092300500000000001, 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544904+0000", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544905+0000", 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7900000000000007e-07 2024-09-15T01:44:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544910+0000", 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.899e-06 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544922+0000", 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2632999999999999e-05 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.544929+0000", 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8600000000000004e-06 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554124+0000", 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091944890000000001 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554134+0000", 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0190000000000001e-05 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=33'8, committed=1, applied=1))", 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.271528185000001, 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088913499999999993, 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545277+0000", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545278+0000", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545288+0000", 2024-09-15T01:44:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7259999999999997e-06 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545300+0000", 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2510000000000001e-05 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545310+0000", 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0049e-05 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554154+0000", 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088435739999999999 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554168+0000", 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4674e-05 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es4 39/34 ECSubWrite(tid=0, reqid=client.4632.0:258, at_version=39'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.288594517, 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096582630000000003, 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528211+0000", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6899999999999998e-07 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528241+0000", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9663999999999999e-05 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528258+0000", 2024-09-15T01:44:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6718000000000002e-05 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.528269+0000", 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1714e-05 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534566+0000", 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062965490000000002 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534578+0000", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1695999999999999e-05 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537839+0000", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003261739 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.537869+0000", 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9714e-05 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.30092973, 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.298691268999999, 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.298121600000002, 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.298062632000001, 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s2 43/34 ECSubWrite(tid=13, reqid=client.4632.0:302, at_version=43'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.296469754, 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094416429999999996, 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520335+0000", 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520336+0000", 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7599999999999997e-07 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520377+0000", 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1192000000000001e-05 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520398+0000", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1013e-05 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520411+0000", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.239e-05 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522362+0000", 2024-09-15T01:44:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001951803 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.522402+0000", 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9764000000000001e-05 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529721+0000", 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007318504 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529777+0000", 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6501e-05 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.310164424, 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.310070646, 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.2434265320000009, 2024-09-15T01:44:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:44:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:494 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~606208] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.2388264170000003, 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010514348, 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 494 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577981+0000", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0770000000000002e-06 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578273+0000", 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000291797 2024-09-15T01:44:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578292+0000", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9491e-05 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578300+0000", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5530000000000004e-06 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578395+0000", 2024-09-15T01:44:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4924000000000001e-05 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578567+0000", 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172413 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.580077+0000", 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001509828 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.581334+0000", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012575889999999999 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588449+0000", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071140400000000003 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588493+0000", 2024-09-15T01:44:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4635999999999998e-05 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:45.931 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:45.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:45.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:45.933 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:45.934 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:45.935 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:45.936 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:45.927+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:46.036 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:46.027+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:46.075 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.075 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:46.075 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:46.075 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:46.134 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:46.127+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:46.185 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:46.185 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:46.240 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:46.236+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:46.300 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.774432697999998, 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:46.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.774393693, 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:46.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.774362748999998, 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.314 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.779478448999999, 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:46.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.777482928000001, 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:46.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.777413396, 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.777390860000001, 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.777323668000001, 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.777254143, 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.777012065000001, 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.776958630999999, 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.776886451999999, 2024-09-15T01:44:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.762924894000001, 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:498 3.4s0 3:25aaa82b:::smithi03327189-9:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.69137396, 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011653834, 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 498 2024-09-15T01:44:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604049+0000", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8739999999999997e-06 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604468+0000", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041931899999999997 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604508+0000", 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9677e-05 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604524+0000", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6030000000000001e-05 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604637+0000", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113602 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604816+0000", 2024-09-15T01:44:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606494+0000", 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016776149999999999 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615523+0000", 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0090292470000000007 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615637+0000", 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011338200000000001 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615699+0000", 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1970999999999993e-05 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:511 3.4s0 3:20577364:::smithi03327189-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6594064209999999, 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014490421999999999, 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 511 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636013+0000", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636015+0000", 2024-09-15T01:44:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2869999999999999e-06 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636020+0000", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.34e-06 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636022+0000", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3779999999999999e-06 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640680+0000", 2024-09-15T01:44:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046585400000000001 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640820+0000", 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014002599999999999 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647530+0000", 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067096409999999997 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647618+0000", 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7530999999999999e-05 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.650503+0000", 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028851409999999999 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.6s1 58/53 ECSubWrite(tid=126, reqid=osd.0.0:126, at_version=58'32, trim_to=0'0, roll_forward_to=57'30))", 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.7265078769999995, 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01167437, 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000002e-07 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568942+0000", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1117e-05 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568954+0000", 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1612e-05 2024-09-15T01:44:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568966+0000", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2051e-05 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569058+0000", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2028999999999996e-05 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569071+0000", 2024-09-15T01:44:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2559e-05 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580549+0000", 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011478647999999999 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580585+0000", 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6050000000000002e-05 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7710357730000004, 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:44:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:44:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:44:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7661189510000002, 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:635 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7626646470000003, 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013058536000000001, 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 635 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532757+0000", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5770000000000001e-06 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532923+0000", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016642800000000001 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532939+0000", 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5574e-05 2024-09-15T01:44:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532947+0000", 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0050000000000004e-06 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536760+0000", 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038128760000000002 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536860+0000", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100008 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538503+0000", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001643451 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541264+0000", 2024-09-15T01:44:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002760701 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545776+0000", 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045126050000000003 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545813+0000", 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6310999999999998e-05 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:639 3.8s1 3:1768942e:::smithi03327189-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7571212420000002, 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011868998, 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 639 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538300+0000", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8199999999999999e-06 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538306+0000", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9689999999999999e-06 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538309+0000", 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4659999999999999e-06 2024-09-15T01:44:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.543951+0000", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056413419999999997 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.544035+0000", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4798999999999999e-05 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.546810+0000", 2024-09-15T01:44:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027750880000000002 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.546900+0000", 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9216999999999996e-05 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.550167+0000", 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032668990000000002 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.408 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:46.410 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:46.404+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:46.510 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:46.503+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:46.517 INFO:tasks.rados.rados.0.smithi033.stdout:255: done (2 left) 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:256: copy_from oid 23 from oid 6 current snap is 18 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:257: read oid 29 snap -1 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:257: expect (ObjNum 114 snap 14 seq_num 114) 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:258: delete oid 45 current snap is 18 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:253: finishing copy_from racing read to smithi03327189-25 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:253: finishing copy_from to smithi03327189-25 2024-09-15T01:44:46.518 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 25 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:44:46.523 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:44:46.535 INFO:tasks.rados.rados.0.smithi033.stdout:256: finishing copy_from racing read to smithi03327189-23 2024-09-15T01:44:46.536 INFO:tasks.rados.rados.0.smithi033.stdout:253: done (4 left) 2024-09-15T01:44:46.536 INFO:tasks.rados.rados.0.smithi033.stdout:258: done (3 left) 2024-09-15T01:44:46.537 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 4 current snap is 18 2024-09-15T01:44:46.537 INFO:tasks.rados.rados.0.smithi033.stdout:259: seq_num 123 ranges {425984=524288,950272=573440,1523712=688128,2211840=294912} 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0086584420000000006, 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0086820460000000006, 2024-09-15T01:44:46.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:44:46.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.531928+0000", 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0059538300000000002, 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060348800000000003, 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.531928+0000", 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.531928+0000", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.531929+0000", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5299999999999999e-07 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.533557+0000", 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001628106 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.533569+0000", 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2206e-05 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.533577+0000", 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6799999999999993e-06 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0054402280000000001, 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055664720000000003, 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:44:46.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.020677846, 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020859572999999999, 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.554 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.020612835999999999, 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020865050999999999, 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:44:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 5 2024-09-15T01:44:46.559 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.559 INFO:tasks.rados.rados.0.smithi033.stdout:259: writing smithi03327189-4 from 425984 to 950272 tid 1 2024-09-15T01:44:46.560 INFO:tasks.rados.rados.0.smithi033.stdout:259: writing smithi03327189-4 from 950272 to 1523712 tid 2 2024-09-15T01:44:46.560 INFO:tasks.rados.rados.0.smithi033.stdout:259: writing smithi03327189-4 from 1523712 to 2211840 tid 3 2024-09-15T01:44:46.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:46.562 INFO:tasks.rados.rados.0.smithi033.stdout:259: writing smithi03327189-4 from 2211840 to 2506752 tid 4 2024-09-15T01:44:46.562 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 1 current snap is 18 2024-09-15T01:44:46.562 INFO:tasks.rados.rados.0.smithi033.stdout:260: seq_num 124 ranges {32768=540672,573440=720896,1294336=786432,2080768=49152} 2024-09-15T01:44:46.562 INFO:tasks.rados.rados.0.smithi033.stdout:260: writing smithi03327189-1 from 32768 to 573440 tid 1 2024-09-15T01:44:46.565 INFO:tasks.rados.rados.0.smithi033.stdout:260: writing smithi03327189-1 from 573440 to 1294336 tid 2 2024-09-15T01:44:46.570 INFO:tasks.rados.rados.0.smithi033.stdout:260: writing smithi03327189-1 from 1294336 to 2080768 tid 3 2024-09-15T01:44:46.570 INFO:tasks.rados.rados.0.smithi033.stdout:260: writing smithi03327189-1 from 2080768 to 2129920 tid 4 2024-09-15T01:44:46.571 INFO:tasks.rados.rados.0.smithi033.stdout:261: snap_remove snap 15 2024-09-15T01:44:46.611 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:46.603+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:46.657 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:46.658 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:46.712 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:46.708+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:46.813 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:46.807+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:46.814 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.290572936, 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:46.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.288087411999999, 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.820 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:46.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:46.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:46.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.28058416, 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.277415869, 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063209149999999999, 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532414+0000", 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532415+0000", 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3200000000000003e-07 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534495+0000", 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020793859999999999 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534525+0000", 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0420999999999999e-05 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534538+0000", 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3022e-05 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534680+0000", 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014239299999999999 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.534711+0000", 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0543999999999998e-05 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538705+0000", 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039944020000000002 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538735+0000", 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9815000000000002e-05 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.164452399, 2024-09-15T01:44:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:46.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:44:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:44:46.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:518 3.es0 3:7de15fb4:::smithi03327189-49:head [setxattr _header (57) in=7b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.160053374, 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063479770000000003, 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 518 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649777+0000", 2024-09-15T01:44:46.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649778+0000", 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.144e-06 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649784+0000", 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8270000000000003e-06 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649794+0000", 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0713999999999999e-05 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649800+0000", 2024-09-15T01:44:46.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2429999999999996e-06 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652010+0000", 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002210231 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652118+0000", 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108608 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.652627+0000", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050885000000000004 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655730+0000", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031025250000000001 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.655776+0000", 2024-09-15T01:44:46.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5812000000000001e-05 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656125+0000", 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034902300000000001 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.160023789, 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:44:46.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:44:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:44:46.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.865 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:549 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1645741980000004, 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072147799999999996, 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 549 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:46.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3900000000000005e-07 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645259+0000", 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8839999999999998e-06 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645268+0000", 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4149999999999999e-06 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645271+0000", 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.841e-06 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645937+0000", 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066678699999999996 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.646068+0000", 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013108799999999999 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.652471+0000", 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064022259999999996 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1645317399999993, 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:44:46.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0590596639999994, 2024-09-15T01:44:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:44:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:44:46.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:44:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0556200899999997, 2024-09-15T01:44:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:44:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:44:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:44:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:565 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~655360] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0512741000000005, 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069304919999999999, 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 565 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758556+0000", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758558+0000", 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9139999999999998e-06 2024-09-15T01:44:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758778+0000", 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000220263 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758805+0000", 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7084e-05 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758816+0000", 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1148999999999999e-05 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759223+0000", 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040634800000000001 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759333+0000", 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110512 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.760874+0000", 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001540689 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764491+0000", 2024-09-15T01:44:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0036169409999999998 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765445+0000", 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00095386899999999998 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765486+0000", 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1723e-05 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:566 3.es0 3:720ccbad:::smithi03327189-29:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0509992199999996, 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071089170000000002, 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 566 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758831+0000", 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758832+0000", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2500000000000001e-06 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758838+0000", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6380000000000001e-06 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758856+0000", 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8034999999999999e-05 2024-09-15T01:44:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758955+0000", 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8777000000000007e-05 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.762040+0000", 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003085951 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.762328+0000", 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028736799999999998 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.763596+0000", 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012677560000000001 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764653+0000", 2024-09-15T01:44:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010569170000000001 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765890+0000", 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001237036 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765940+0000", 2024-09-15T01:44:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0188999999999997e-05 2024-09-15T01:44:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0508341489999999, 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:44:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.29267653199999999, 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:44:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:44:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.29262561199999998, 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:44:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.292560602, 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:44:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:44:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.28060620800000002, 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088343680000000004, 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:44:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538058+0000", 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005438109 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.27738799400000003, 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:44:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.27334392800000001, 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:44:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:46.935 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:46.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:46.938 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:46.931+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:47.038 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:47.031+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:47.046 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:47.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:47.139 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:47.131+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:47.154 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:47.155 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:47.155 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:47.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:47.240 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:47.236+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:47.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.751039722000002, 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:47.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.750985483000001, 2024-09-15T01:44:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.750966345000002, 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:47.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.741991445, 2024-09-15T01:44:47.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:47.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:47.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.741923783000001, 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:47.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.741898041999999, 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.728827233000001, 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:47.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:47.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:47.301 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.727785328, 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.726964522999999, 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:47.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:47.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.730088665, 2024-09-15T01:44:47.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084277399999999995, 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530349+0000", 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530350+0000", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530476+0000", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012622100000000001 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530495+0000", 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8326e-05 2024-09-15T01:44:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530506+0000", 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1063999999999999e-05 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530816+0000", 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030996700000000002 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530831+0000", 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5129e-05 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538759+0000", 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079275890000000005 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538777+0000", 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 57/53 ECSubWrite(tid=20, reqid=client.4632.0:477, at_version=57'78, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.727788425, 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089560869999999997, 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532650+0000", 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532651+0000", 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5239999999999999e-06 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532968+0000", 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031664399999999999 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.532988+0000", 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0676e-05 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533000+0000", 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1318000000000001e-05 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533071+0000", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0779999999999997e-05 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.533082+0000", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1447000000000001e-05 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541581+0000", 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084990299999999994 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.541606+0000", 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4668000000000002e-05 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:481 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~475136] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.725001312, 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095211319999999999, 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 481 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535439+0000", 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5320000000000001e-06 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535767+0000", 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032738100000000002 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535789+0000", 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1917e-05 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535882+0000", 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3171000000000006e-05 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535944+0000", 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2124000000000004e-05 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.536068+0000", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012412600000000001 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.537727+0000", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016587010000000001 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544825+0000", 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070983599999999997 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544893+0000", 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8188999999999994e-05 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544958+0000", 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4630999999999998e-05 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:482 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~442368] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.721290323, 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087432670000000007, 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 482 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539148+0000", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539150+0000", 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1459999999999999e-06 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539423+0000", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000273195 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539441+0000", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7725999999999999e-05 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539449+0000", 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7700000000000001e-06 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539516+0000", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7158999999999999e-05 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.539662+0000", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000145861 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.540574+0000", 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091271500000000003 2024-09-15T01:44:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544826+0000", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042521360000000001 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547817+0000", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029910129999999998 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.547891+0000", 2024-09-15T01:44:47.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3546000000000004e-05 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.4s3 58/53 ECSubWrite(tid=112, reqid=osd.1.0:112, at_version=58'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6907779129999998, 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087782980000000004, 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569660+0000", 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2300000000000002e-07 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569683+0000", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2900000000000001e-05 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569693+0000", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569698+0000", 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2329999999999997e-06 2024-09-15T01:44:47.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.571690+0000", 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001991788 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.571705+0000", 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4713e-05 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.578409+0000", 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067040889999999999 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.578438+0000", 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9844000000000001e-05 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6366281180000009, 2024-09-15T01:44:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:44:47.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.339 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:537 3.bs0 3:dc585df2:::smithi03327189-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6365620419999996, 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011082247999999999, 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 537 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623877+0000", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623881+0000", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623895+0000", 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3091000000000001e-05 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623902+0000", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.153e-06 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624357+0000", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045563899999999998 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624496+0000", 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013903799999999999 2024-09-15T01:44:47.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634734+0000", 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01023717 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634849+0000", 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001154 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634958+0000", 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109345 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 58/53 ECSubWrite(tid=36, reqid=client.4632.0:528, at_version=58'91, trim_to=0'0, roll_forward_to=58'89))", 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6324083280000004, 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085050799999999999, 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628030+0000", 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2399999999999998e-07 2024-09-15T01:44:47.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628216+0000", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018574499999999999 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628228+0000", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1625e-05 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628234+0000", 2024-09-15T01:44:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1299999999999998e-06 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628280+0000", 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5904e-05 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.628290+0000", 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0180000000000001e-05 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.636504+0000", 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082141460000000003 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.636535+0000", 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0725999999999997e-05 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:555 3.bs0 3:d32e3b1b:::smithi03327189-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6002654760000006, 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094173030000000001, 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 555 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660174+0000", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660177+0000", 2024-09-15T01:44:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7069999999999998e-06 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660199+0000", 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1659999999999999e-05 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660205+0000", 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7869999999999998e-06 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663351+0000", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031458609999999998 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663435+0000", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3947000000000001e-05 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668442+0000", 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050072629999999996 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668572+0000", 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013023000000000001 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.669590+0000", 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010177529999999999 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6573195070000004, 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:44:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.361 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6571860420000002, 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:47.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:44:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:44:47.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:44:47.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:47.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:47.368 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:47.368 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:47.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:47.370 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:47.367+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:47.470 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:47.464+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:47.483 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:47.483 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:47.483 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:47.483 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:47.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:47.514 INFO:tasks.rados.rados.0.smithi033.stdout:261: done (5 left) 2024-09-15T01:44:47.514 INFO:tasks.rados.rados.0.smithi033.stdout:262: snap_create 2024-09-15T01:44:47.514 INFO:tasks.rados.rados.0.smithi033.stdout:256: finishing copy_from to smithi03327189-23 2024-09-15T01:44:47.514 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 23 v 66 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:44:47.515 INFO:tasks.rados.rados.0.smithi033.stdout:257: expect (ObjNum 114 snap 14 seq_num 114) 2024-09-15T01:44:47.571 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:47.564+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:47.600 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:47.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:47.600 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:47.601 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:47.601 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:47.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:47.673 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:47.667+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:47.675 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:47.676 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:47.676 INFO:tasks.thrashosds.thrasher:inject_pause on osd.8 2024-09-15T01:44:47.676 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T01:44:47.676 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T01:44:47.676 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok config set filestore_inject_stall 3 2024-09-15T01:44:47.689 INFO:tasks.rados.rados.0.smithi033.stdout:254: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:44:47.690 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 1 to smithi03327189-4 2024-09-15T01:44:47.690 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 2 to smithi03327189-4 2024-09-15T01:44:47.690 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 3 to smithi03327189-4 2024-09-15T01:44:47.690 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 4 to smithi03327189-4 2024-09-15T01:44:47.690 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 5 to smithi03327189-4 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:259: finishing write tid 6 to smithi03327189-4 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 72 (ObjNum 123 snap 18 seq_num 123) dirty exists 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:259: left oid 4 (ObjNum 123 snap 18 seq_num 123) 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 1 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 2 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 3 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 4 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 5 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: finishing write tid 6 to smithi03327189-1 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 1 v 58 (ObjNum 124 snap 18 seq_num 124) dirty exists 2024-09-15T01:44:47.691 INFO:tasks.rados.rados.0.smithi033.stdout:260: left oid 1 (ObjNum 124 snap 18 seq_num 124) 2024-09-15T01:44:47.711 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:47.711 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.187747688999998, 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:47.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:44:47.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:47.717 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.187689343000002, 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:44:47.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:44:47.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:44:47.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.722 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.187610585999998, 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.727 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.728 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.729 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.187430585000001, 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.731 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.732 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.733 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.734 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.736 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:47.737 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 11.177120629999999, 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:47.738 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:44:47.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.741 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.743 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.121874884, 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.745 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:44:47.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:44:47.747 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.748 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:44:47.749 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:47.750 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.113032815, 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.751 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:44:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.752 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.755 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:44:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:44:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.756 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:44:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:44:47.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.759 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:44:47.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.761 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.082153194, 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:44:47.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:44:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.767 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.768 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:44:47.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.771 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.772 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.773 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.082073826, 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.774 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:44:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.776 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:44:47.778 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.062183445000001, 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:44:47.781 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.782 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:44:47.783 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:44:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:44:47.784 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:44:47.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.786 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.060820783, 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.788 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.789 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:44:47.791 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.792 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.060806258, 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.795 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.796 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.797 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:47.798 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.799 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:44:47.801 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.802 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:44:47.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:547 3.0s0 3:08c156cd:::smithi03327189-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.060757836000001, 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0091027969999999993, 2024-09-15T01:44:47.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 547 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.805 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:47.806 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645643+0000", 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.79e-07 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.807 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645644+0000", 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.699e-06 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645650+0000", 2024-09-15T01:44:47.808 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.942e-06 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645653+0000", 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.452e-06 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.809 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653028+0000", 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073755519999999996 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653094+0000", 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5718999999999999e-05 2024-09-15T01:44:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654745+0000", 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016511539999999999 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.038420497000001, 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.812 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:47.813 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.814 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.816 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.817 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.818 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:560 3.0s0 3:0899e339:::smithi03327189-38:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.034688048, 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090281700000000003, 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.820 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 560 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671713+0000", 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671717+0000", 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1749999999999998e-06 2024-09-15T01:44:47.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671737+0000", 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.948e-05 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671744+0000", 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9569999999999996e-06 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.823 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676355+0000", 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046115949999999996 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676528+0000", 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017328999999999999 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677261+0000", 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073246900000000002 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677982+0000", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072084199999999997 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680660+0000", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002678541 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680740+0000", 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9963000000000007e-05 2024-09-15T01:44:47.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.034650174999999, 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.826 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:44:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.829 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.189525333, 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:44:47.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.833 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:44:47.834 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:47.835 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:44:47.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:44:47.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:675 3.cs0 3:3584cd82:::smithi03327189-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1894082930000001, 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017344149999999999, 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 675 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516993+0000", 2024-09-15T01:44:47.840 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5600000000000001e-06 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517001+0000", 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517020+0000", 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9364999999999998e-05 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.841 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517028+0000", 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6229999999999997e-06 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517199+0000", 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017127399999999999 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7309999999999998e-06 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.842 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524428+0000", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072231680000000003 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524522+0000", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4467000000000002e-05 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534164+0000", 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0096419079999999994 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534243+0000", 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9296999999999997e-05 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534336+0000", 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2608000000000005e-05 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:680 3.0s0 3:0a6b0087:::smithi03327189-4:head [append 0~524288] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1632899999999999, 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010598392999999999, 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 680 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.845 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543113+0000", 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5490000000000001e-06 2024-09-15T01:44:47.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544026+0000", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091366600000000004 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544053+0000", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7015000000000001e-05 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544068+0000", 2024-09-15T01:44:47.847 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4977000000000001e-05 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544122+0000", 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3804000000000002e-05 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544197+0000", 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5139e-05 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.848 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.545372+0000", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001175024 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.547186+0000", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001814026 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553517+0000", 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063304659999999999 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553709+0000", 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019172699999999999 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:685 3.0s0 3:0a6b0087:::smithi03327189-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.1496443350000001, 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01007038, 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:47.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 685 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.851 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8299999999999997e-07 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556760+0000", 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:44:47.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556766+0000", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556770+0000", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6109999999999998e-06 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.562875+0000", 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061050740000000003 2024-09-15T01:44:47.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.563061+0000", 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000186066 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564501+0000", 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014400400000000001 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:47.854 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564606+0000", 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010522399999999999 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.566826+0000", 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002219782 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:47.855 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:47.857 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:47.857 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T01:44:47.858 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:47.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:47.858 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:47.851+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:47.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:47.965 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:47.965 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:47.965 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:47.965 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:47.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:47.984 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:47.980+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:48.059 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:48.051+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:48.077 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:48.078 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:48.078 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:48.078 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:48.078 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:48.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:48.092 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:48.113 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:48.134 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:48.156 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:48.159 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:48.152+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:48.178 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:48.191 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 51.791163812000001, 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006287684, 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 5 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394852+0000", 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7460000000000001e-06 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394870+0000", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8074e-05 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394905+0000", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4876000000000003e-05 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394923+0000", 2024-09-15T01:44:48.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7691000000000001e-05 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395069+0000", 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014603000000000001 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395085+0000", 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6143000000000001e-05 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398289+0000", 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003203759 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398489+0000", 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000200223 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:48.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398670+0000", 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000180359 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.400465+0000", 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001795718 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401043+0000", 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057742000000000004 2024-09-15T01:44:48.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401080+0000", 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7283000000000002e-05 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401137+0000", 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7361999999999997e-05 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 50.776266401999997, 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0068396289999999998, 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.199 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 7 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:48.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409750+0000", 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0699999999999998e-06 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409763+0000", 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.325e-05 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409805+0000", 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1412000000000003e-05 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409839+0000", 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3692999999999997e-05 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410004+0000", 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016576300000000001 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410032+0000", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8076e-05 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413650+0000", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036180219999999998 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413889+0000", 2024-09-15T01:44:48.203 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00023881400000000001 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.414266+0000", 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00037692599999999998 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.415749+0000", 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001483301 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.204 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416329+0000", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057982400000000005 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416478+0000", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014846299999999999 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416587+0000", 2024-09-15T01:44:48.205 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109015 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:222 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.667754066999997, 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026140312999999998, 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.206 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 222 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518262+0000", 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1090000000000001e-06 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518270+0000", 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.208 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518286+0000", 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6081000000000001e-05 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518298+0000", 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1941e-05 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518644+0000", 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034667699999999999 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518650+0000", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526775+0000", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081258210000000001 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526967+0000", 2024-09-15T01:44:48.210 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191991 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.544400+0000", 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017432277999999999 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:223 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:48.211 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.667692154999997, 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027026921999999998, 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 223 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518322+0000", 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:44:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518328+0000", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4210000000000003e-06 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518338+0000", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.022e-05 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518343+0000", 2024-09-15T01:44:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.66e-06 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518713+0000", 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036931399999999998 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518715+0000", 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.841e-06 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527539+0000", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0088243579999999992 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527752+0000", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000212888 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017596414000000001 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:224 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.667659202999999, 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028715145000000001, 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 224 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518355+0000", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518359+0000", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9550000000000002e-06 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518367+0000", 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:44:48.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518372+0000", 2024-09-15T01:44:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8420000000000004e-06 2024-09-15T01:44:48.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518770+0000", 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039736300000000002 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518772+0000", 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2019999999999998e-06 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528208+0000", 2024-09-15T01:44:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094362790000000005 2024-09-15T01:44:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528403+0000", 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019513399999999999 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.223 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.547069+0000", 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018666242999999999 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.9s4 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.658462452999999, 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0062097970000000004, 2024-09-15T01:44:48.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:48.226 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527552+0000", 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5199999999999996e-07 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.228 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528961+0000", 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014095609999999999 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528973+0000", 2024-09-15T01:44:48.229 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1335000000000001e-05 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528990+0000", 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7359000000000001e-05 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529050+0000", 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0235000000000002e-05 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.231 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529062+0000", 2024-09-15T01:44:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1531e-05 2024-09-15T01:44:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.232 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533731+0000", 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046689110000000004 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533761+0000", 2024-09-15T01:44:48.233 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0113000000000001e-05 2024-09-15T01:44:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/32 ECSubWriteReply(tid=61, last_complete=33'22, committed=1, applied=1))", 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.658016648999997, 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050195400000000003, 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.236 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527998+0000", 2024-09-15T01:44:48.237 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0839999999999999e-06 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528005+0000", 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7209999999999997e-06 2024-09-15T01:44:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528018+0000", 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3818000000000001e-05 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528026+0000", 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1359999999999999e-06 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533000+0000", 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049747359999999996 2024-09-15T01:44:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533016+0000", 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6045e-05 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:255 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 29.665357604, 2024-09-15T01:44:48.241 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049454870000000001, 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 255 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520660+0000", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6600000000000001e-06 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520673+0000", 2024-09-15T01:44:48.243 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3406e-05 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520707+0000", 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3902999999999997e-05 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520727+0000", 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0588999999999999e-05 2024-09-15T01:44:48.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520839+0000", 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011144199999999999 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.521184+0000", 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034537000000000001 2024-09-15T01:44:48.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.525601+0000", 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044171169999999999 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:256 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 29.665238899999999, 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061411549999999997, 2024-09-15T01:44:48.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 256 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520776+0000", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5e-06 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520786+0000", 2024-09-15T01:44:48.248 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520807+0000", 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1169999999999999e-05 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520819+0000", 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1589e-05 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522420+0000", 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001600953 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.250 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522494+0000", 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4438999999999997e-05 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.526916+0000", 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044215110000000004 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:257 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:48.251 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 29.665178805, 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006854008, 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 257 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.252 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520836+0000", 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520844+0000", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8599999999999993e-06 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520862+0000", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7859999999999998e-05 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520872+0000", 2024-09-15T01:44:48.254 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522823+0000", 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019506020000000001 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522947+0000", 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012457899999999999 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.527689+0000", 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004741398 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(osd.1.17:1 3.0s0 3:08c156cd:::smithi03327189-34:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.660085507000002, 2024-09-15T01:44:48.256 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012067648, 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525932+0000", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.693e-06 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525943+0000", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0917000000000001e-05 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525964+0000", 2024-09-15T01:44:48.258 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0993e-05 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525975+0000", 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1823e-05 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527736+0000", 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001760345 2024-09-15T01:44:48.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527967+0000", 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230769 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.537996+0000", 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010029108 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=50, reqid=client.4632.0:290, at_version=43'17, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.652821540000001, 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061827899999999996, 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.261 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.68e-07 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533416+0000", 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000223594 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533428+0000", 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649e-05 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.263 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.534981+0000", 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001553029 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.264 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535038+0000", 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6560000000000001e-05 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535050+0000", 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1885999999999999e-05 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539340+0000", 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004289836 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539375+0000", 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5067999999999997e-05 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=51, reqid=client.4632.0:291, at_version=43'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.650980365999999, 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049865860000000003, 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.267 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9200000000000002e-07 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535160+0000", 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126678 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.268 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535171+0000", 2024-09-15T01:44:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0783e-05 2024-09-15T01:44:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.269 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535178+0000", 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9809999999999997e-06 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.270 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535795+0000", 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061714799999999998 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535804+0000", 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0680000000000003e-06 2024-09-15T01:44:48.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539989+0000", 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041847079999999997 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.272 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.540020+0000", 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0927999999999999e-05 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:393 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 0] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 21.678851839, 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077679840000000003, 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 393 2024-09-15T01:44:48.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507165+0000", 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2729999999999998e-06 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507179+0000", 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4344e-05 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507215+0000", 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5509000000000002e-05 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.278 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507230+0000", 2024-09-15T01:44:48.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5116000000000001e-05 2024-09-15T01:44:48.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.279 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.279 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507374+0000", 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014417100000000001 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507400+0000", 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6222000000000001e-05 2024-09-15T01:44:48.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511249+0000", 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038487730000000002 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.281 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511443+0000", 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194234 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.514930+0000", 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034863419999999999 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.282 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:394 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 21.678758600999998, 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081504530000000002, 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 394 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507256+0000", 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1230000000000001e-06 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507260+0000", 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3970000000000004e-06 2024-09-15T01:44:48.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507278+0000", 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8077e-05 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507285+0000", 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507540+0000", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000254541 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507553+0000", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2962999999999999e-05 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511752+0000", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041991509999999999 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:48.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511891+0000", 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013930299999999999 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515075+0000", 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0031832549999999999 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515236+0000", 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000161096 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515405+0000", 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016956100000000001 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:396 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~671744] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:48.289 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:48.290 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.669484911000001, 2024-09-15T01:44:48.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014871097, 2024-09-15T01:44:48.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:48.290 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 396 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:48.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516531+0000", 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8700000000000001e-06 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517417+0000", 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00088526499999999999 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517445+0000", 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8351e-05 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517458+0000", 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2894000000000001e-05 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517618+0000", 2024-09-15T01:44:48.295 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000160025 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517871+0000", 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025296800000000001 2024-09-15T01:44:48.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.520096+0000", 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022246259999999999 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523463+0000", 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0033674690000000001 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531316+0000", 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078531639999999993 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531400+0000", 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3465000000000001e-05 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:397 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~770048] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.662137455, 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0082945069999999996, 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:48.302 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.302 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.302 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 397 2024-09-15T01:44:48.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523879+0000", 2024-09-15T01:44:48.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2610000000000002e-06 2024-09-15T01:44:48.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525106+0000", 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266060000000001 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.307 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525135+0000", 2024-09-15T01:44:48.307 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:44:48.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.307 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525148+0000", 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3892e-05 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525342+0000", 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019338100000000001 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525610+0000", 2024-09-15T01:44:48.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026807099999999999 2024-09-15T01:44:48.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.528197+0000", 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025871959999999999 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531425+0000", 2024-09-15T01:44:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032278430000000002 2024-09-15T01:44:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:48.312 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532119+0000", 2024-09-15T01:44:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069357900000000005 2024-09-15T01:44:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532171+0000", 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2008000000000001e-05 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:398 3.5s0 3:ac2d3331:::smithi03327189-18:head [setxattr _header (57) in=7b] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.660844074, 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073765130000000003, 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:48.316 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.316 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.316 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 398 2024-09-15T01:44:48.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.317 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:48.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525171+0000", 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.320 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.320 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525178+0000", 2024-09-15T01:44:48.320 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1239999999999999e-06 2024-09-15T01:44:48.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525193+0000", 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.535e-05 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525203+0000", 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4749999999999999e-06 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530018+0000", 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048148799999999997 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530132+0000", 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000114434 2024-09-15T01:44:48.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530708+0000", 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057595400000000005 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531935+0000", 2024-09-15T01:44:48.324 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266790000000001 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532465+0000", 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00052987900000000003 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532546+0000", 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1151999999999998e-05 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:399 3.5s0 3:ac2d3331:::smithi03327189-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.660489603999999, 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090256980000000004, 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 399 2024-09-15T01:44:48.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525525+0000", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5009999999999999e-06 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525531+0000", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1120000000000001e-06 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525550+0000", 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8819000000000001e-05 2024-09-15T01:44:48.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525559+0000", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.833e-06 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531733+0000", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061738590000000003 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531869+0000", 2024-09-15T01:44:48.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013559600000000001 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532982+0000", 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011133320000000001 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.533176+0000", 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019444500000000001 2024-09-15T01:44:48.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.534550+0000", 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013732009999999999 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 48/34 ECSubWriteReply(tid=157, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:48.331 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.660415643, 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005543699, 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:48.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525599+0000", 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525606+0000", 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7349999999999999e-06 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525621+0000", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5559000000000001e-05 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525628+0000", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6560000000000003e-06 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531120+0000", 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0054919829999999998 2024-09-15T01:44:48.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531142+0000", 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1671e-05 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:48.335 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:48.337 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:48.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:48.338 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:48.331+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:48.344 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:48.437 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:48.432+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:48.449 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:48.449 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:48.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:48.518 INFO:tasks.rados.rados.0.smithi033.stdout:254: done (5 left) 2024-09-15T01:44:48.518 INFO:tasks.rados.rados.0.smithi033.stdout:256: done (4 left) 2024-09-15T01:44:48.518 INFO:tasks.rados.rados.0.smithi033.stdout:257: done (3 left) 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:259: done (2 left) 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:260: done (1 left) 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:262: done (0 left) 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:263: rollback oid 33 current snap is 19 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 33 to 18 2024-09-15T01:44:48.520 INFO:tasks.rados.rados.0.smithi033.stdout:264: delete oid 44 current snap is 19 2024-09-15T01:44:48.524 INFO:tasks.rados.rados.0.smithi033.stdout:264: done (1 left) 2024-09-15T01:44:48.524 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 29 current snap is 19 2024-09-15T01:44:48.525 INFO:tasks.rados.rados.0.smithi033.stdout:265: seq_num 126 ranges {4243456=573440,4816896=622592,5439488=622592,6062080=311296} 2024-09-15T01:44:48.538 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:48.531+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:48.556 INFO:tasks.rados.rados.0.smithi033.stdout:265: writing smithi03327189-29 from 4243456 to 4816896 tid 1 2024-09-15T01:44:48.560 INFO:tasks.rados.rados.0.smithi033.stdout:265: writing smithi03327189-29 from 4816896 to 5439488 tid 2 2024-09-15T01:44:48.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:44:48.563 INFO:tasks.rados.rados.0.smithi033.stdout:265: writing smithi03327189-29 from 5439488 to 6062080 tid 3 2024-09-15T01:44:48.566 INFO:tasks.rados.rados.0.smithi033.stdout:265: writing smithi03327189-29 from 6062080 to 6373376 tid 4 2024-09-15T01:44:48.566 INFO:tasks.rados.rados.0.smithi033.stdout:266: copy_from oid 39 from oid 24 current snap is 19 2024-09-15T01:44:48.566 INFO:tasks.rados.rados.0.smithi033.stdout:263: finishing rollback tid 0 to smithi03327189-33 2024-09-15T01:44:48.566 INFO:tasks.rados.rados.0.smithi033.stdout:263: finishing rollback tid 1 to smithi03327189-33 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:263: finishing rollback tid 2 to smithi03327189-33 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 33 v 34 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:263: done (2 left) 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:267: read oid 32 snap -1 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:267: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:268: snap_create 2024-09-15T01:44:48.567 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 1 to smithi03327189-29 2024-09-15T01:44:48.567 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:48.567 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:48.572 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 2 to smithi03327189-29 2024-09-15T01:44:48.574 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 3 to smithi03327189-29 2024-09-15T01:44:48.575 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 4 to smithi03327189-29 2024-09-15T01:44:48.577 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:48.577 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 5 to smithi03327189-29 2024-09-15T01:44:48.580 INFO:tasks.rados.rados.0.smithi033.stdout:265: finishing write tid 6 to smithi03327189-29 2024-09-15T01:44:48.580 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 29 v 33 (ObjNum 126 snap 19 seq_num 126) dirty exists 2024-09-15T01:44:48.580 INFO:tasks.rados.rados.0.smithi033.stdout:265: left oid 29 (ObjNum 126 snap 19 seq_num 126) 2024-09-15T01:44:48.582 INFO:tasks.rados.rados.0.smithi033.stdout:267: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:48.639 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:48.635+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:48.645 INFO:tasks.rados.rados.0.smithi033.stdout:266: finishing copy_from racing read to smithi03327189-39 2024-09-15T01:44:48.645 INFO:tasks.rados.rados.0.smithi033.stdout:266: finishing copy_from to smithi03327189-39 2024-09-15T01:44:48.645 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 39 v 52 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-09-15T01:44:48.671 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:48.671 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:48.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:48.740 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:48.731+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:48.783 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:48.783 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:48.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:48.841 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:48.835+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:44:48.876+0000 7f95067fc700 0 --2- 172.21.15.33:0/2449745290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95080b3890 0x7f95081465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:44:48.895 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:48.895 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:48.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:48.942 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:48.935+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:49.001 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.001 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.011 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:49.043 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:49.036+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:49.117 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.118 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:49.143 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:49.140+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:49.233 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.233 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.242 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:49.244 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:49.240+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:49.337 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.338 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.345 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:49.340+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:49.346 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:49.444 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.444 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.445 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:49.439+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:49.453 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:49.521 INFO:tasks.rados.rados.0.smithi033.stdout:265: done (3 left) 2024-09-15T01:44:49.521 INFO:tasks.rados.rados.0.smithi033.stdout:266: done (2 left) 2024-09-15T01:44:49.521 INFO:tasks.rados.rados.0.smithi033.stdout:267: done (1 left) 2024-09-15T01:44:49.521 INFO:tasks.rados.rados.0.smithi033.stdout:268: done (0 left) 2024-09-15T01:44:49.522 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 16 current snap is 20 2024-09-15T01:44:49.522 INFO:tasks.rados.rados.0.smithi033.stdout:269: seq_num 127 ranges {262144=589824,851968=753664,1605632=475136,2080768=327680} 2024-09-15T01:44:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:44:49.532 INFO:tasks.rados.rados.0.smithi033.stdout:269: writing smithi03327189-16 from 262144 to 851968 tid 1 2024-09-15T01:44:49.535 INFO:tasks.rados.rados.0.smithi033.stdout:269: writing smithi03327189-16 from 851968 to 1605632 tid 2 2024-09-15T01:44:49.538 INFO:tasks.rados.rados.0.smithi033.stdout:269: writing smithi03327189-16 from 1605632 to 2080768 tid 3 2024-09-15T01:44:49.541 INFO:tasks.rados.rados.0.smithi033.stdout:269: writing smithi03327189-16 from 2080768 to 2408448 tid 4 2024-09-15T01:44:49.541 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 29 current snap is 20 2024-09-15T01:44:49.541 INFO:tasks.rados.rados.0.smithi033.stdout:270: seq_num 128 ranges {6373376=606208,6979584=507904,7487488=573440,8060928=475136} 2024-09-15T01:44:49.547 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:49.539+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:49.562 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.563 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.572 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:49.582 INFO:tasks.rados.rados.0.smithi033.stdout:270: writing smithi03327189-29 from 6373376 to 6979584 tid 1 2024-09-15T01:44:49.585 INFO:tasks.rados.rados.0.smithi033.stdout:270: writing smithi03327189-29 from 6979584 to 7487488 tid 2 2024-09-15T01:44:49.589 INFO:tasks.rados.rados.0.smithi033.stdout:270: writing smithi03327189-29 from 7487488 to 8060928 tid 3 2024-09-15T01:44:49.593 INFO:tasks.rados.rados.0.smithi033.stdout:270: writing smithi03327189-29 from 8060928 to 8536064 tid 4 2024-09-15T01:44:49.593 INFO:tasks.rados.rados.0.smithi033.stdout:271: snap_create 2024-09-15T01:44:49.593 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 1 to smithi03327189-16 2024-09-15T01:44:49.593 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 2 to smithi03327189-16 2024-09-15T01:44:49.594 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 3 to smithi03327189-16 2024-09-15T01:44:49.594 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 4 to smithi03327189-16 2024-09-15T01:44:49.594 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 5 to smithi03327189-16 2024-09-15T01:44:49.594 INFO:tasks.rados.rados.0.smithi033.stdout:269: finishing write tid 6 to smithi03327189-16 2024-09-15T01:44:49.594 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 16 v 16 (ObjNum 127 snap 20 seq_num 127) dirty exists 2024-09-15T01:44:49.595 INFO:tasks.rados.rados.0.smithi033.stdout:269: left oid 16 (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:44:49.595 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 1 to smithi03327189-29 2024-09-15T01:44:49.595 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 2 to smithi03327189-29 2024-09-15T01:44:49.598 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 3 to smithi03327189-29 2024-09-15T01:44:49.599 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 4 to smithi03327189-29 2024-09-15T01:44:49.601 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 5 to smithi03327189-29 2024-09-15T01:44:49.604 INFO:tasks.rados.rados.0.smithi033.stdout:270: finishing write tid 6 to smithi03327189-29 2024-09-15T01:44:49.604 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 29 v 39 (ObjNum 128 snap 20 seq_num 128) dirty exists 2024-09-15T01:44:49.604 INFO:tasks.rados.rados.0.smithi033.stdout:270: left oid 29 (ObjNum 128 snap 20 seq_num 128) 2024-09-15T01:44:49.648 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:49.643+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:49.666 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.666 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.676 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:49.749 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:49.744+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:49.826 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.826 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.836 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:49.851 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:49.843+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:49.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:49.952 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:49.943+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:50.044 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:50.044 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:50.053 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:50.047+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:50.054 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:50.152 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:50.153 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:50.154 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:50.147+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:50.161 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:50.256 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:50.251+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:50.264 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:50.265 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:50.273 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:50.356 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:50.351+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:50.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:50.372 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:50.372 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:50.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:50.458 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:50.455+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:50.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:50.508 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:50.509 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:50.509 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:50.509 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:50.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:269: done (2 left) 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:270: done (1 left) 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:271: done (0 left) 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:272: copy_from oid 20 from oid 46 current snap is 21 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:273: copy_from oid 35 from oid 14 current snap is 21 2024-09-15T01:44:50.523 INFO:tasks.rados.rados.0.smithi033.stdout:274: setattr oid 33 current snap is 21 2024-09-15T01:44:50.523 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:50.524 INFO:tasks.rados.rados.0.smithi033.stdout:275: setattr oid 27 current snap is 21 2024-09-15T01:44:50.527 INFO:tasks.rados.rados.0.smithi033.stdout:276: rollback oid 15 current snap is 21 2024-09-15T01:44:50.527 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 15 to 20 2024-09-15T01:44:50.527 INFO:tasks.rados.rados.0.smithi033.stdout:277: read oid 2 snap 19 2024-09-15T01:44:50.527 INFO:tasks.rados.rados.0.smithi033.stdout:277: expect (ObjNum 93 snap 11 seq_num 93) 2024-09-15T01:44:50.527 INFO:tasks.rados.rados.0.smithi033.stdout:278: snap_remove snap 16 2024-09-15T01:44:50.555 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:50.560 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:50.555+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:50.577 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:50.597 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:50.625 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:50.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:50.645 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:50.661 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:50.656+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:50.678 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:50.715 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:50.745 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:50.752 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.221107287999999, 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:50.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:50.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.21867589, 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:50.766 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.218437074999997, 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:50.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.772 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.218405691999997, 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.239198262000002, 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.231283859999998, 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.231254597000003, 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 33/32 ECSubWriteReply(tid=29, last_complete=29'1, committed=1, applied=1))", 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.203817751000003, 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010365101, 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543737+0000", 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543738+0000", 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.11e-06 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543744+0000", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5999999999999997e-06 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543756+0000", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.543763+0000", 2024-09-15T01:44:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8469999999999996e-06 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554089+0000", 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010325641 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554102+0000", 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3003000000000001e-05 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.231679278000001, 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.229440817, 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.228871148, 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.228812179999998, 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.240913972000001, 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.240820194000001, 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.174176080000001, 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:44:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:44:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:494 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~606208] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.169575965, 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010514348, 2024-09-15T01:44:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 494 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577979+0000", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.577981+0000", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0770000000000002e-06 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578273+0000", 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000291797 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578292+0000", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9491e-05 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578300+0000", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5530000000000004e-06 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578395+0000", 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4924000000000001e-05 2024-09-15T01:44:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578567+0000", 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172413 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.580077+0000", 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001509828 2024-09-15T01:44:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.581334+0000", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012575889999999999 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588449+0000", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071140400000000003 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.588493+0000", 2024-09-15T01:44:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4635999999999998e-05 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=117, reqid=osd.4.0:117, at_version=68'74, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.2330848030000001, 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011721416, 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514471+0000", 2024-09-15T01:44:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514541+0000", 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0955000000000001e-05 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514567+0000", 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.597e-05 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514584+0000", 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6463000000000001e-05 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514727+0000", 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014272899999999999 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514754+0000", 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7290000000000001e-05 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526159+0000", 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011404621 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526191+0000", 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2840999999999999e-05 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/53 ECSubWrite(tid=118, reqid=osd.4.0:118, at_version=68'50, trim_to=0'0, roll_forward_to=67'48))", 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.2326775539999999, 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012184878999999999, 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514879+0000", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514938+0000", 2024-09-15T01:44:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9142999999999999e-05 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514963+0000", 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5165999999999999e-05 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515032+0000", 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8602999999999994e-05 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515160+0000", 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012877200000000001 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515188+0000", 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7313000000000001e-05 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527007+0000", 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011819079999999999 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527062+0000", 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5328999999999998e-05 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=68'76, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.230623171, 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014580965, 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516933+0000", 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6249999999999999e-06 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516977+0000", 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3551999999999999e-05 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517002+0000", 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4762000000000001e-05 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517016+0000", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.401e-05 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517789+0000", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077353199999999995 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517813+0000", 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3748e-05 2024-09-15T01:44:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531475+0000", 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013662636000000001 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531513+0000", 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7100000000000001e-05 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:4 3.1s0 3:85bd41e4:::smithi03327189-14:head [assert-version v38,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.223275902, 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010562561999999999, 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524282+0000", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524293+0000", 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1588e-05 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524323+0000", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9383e-05 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524339+0000", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6314e-05 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524415+0000", 2024-09-15T01:44:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5636000000000006e-05 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524530+0000", 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115263 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.534841+0000", 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010311512 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:50.865 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:50.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:50.867 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:50.867 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:50.869 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:50.863+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:50.967 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:50.960+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:51.016 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:51.017 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:51.072 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:51.068+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:51.125 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.125 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:51.125 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:51.125 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:51.126 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:51.168 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:51.163+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:51.241 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.241 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:228 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.715072669999998, 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011558835, 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 228 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520986+0000", 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520988+0000", 2024-09-15T01:44:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.885e-06 2024-09-15T01:44:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.520994+0000", 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1009999999999997e-06 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521006+0000", 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1473e-05 2024-09-15T01:44:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521015+0000", 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3000000000000007e-06 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523363+0000", 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023486050000000001 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.523711+0000", 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034738800000000002 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.532545+0000", 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088340829999999995 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.715033665, 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.715002720999998, 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s5 37/34 ECSubWrite(tid=92, reqid=osd.5.0:92, at_version=37'30, trim_to=0'0, roll_forward_to=35'28))", 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.720118421000002, 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011325457000000001, 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515940+0000", 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.515941+0000", 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8999999999999995e-07 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516002+0000", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0457000000000001e-05 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516032+0000", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9980000000000001e-05 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516050+0000", 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8070000000000001e-05 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516239+0000", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018969500000000001 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.516266+0000", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6359999999999998e-05 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527230+0000", 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010964685 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:16.527266+0000", 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5420000000000003e-05 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 32.718122899999997, 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 32.718053368, 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 32.718030831999997, 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.717963640000001, 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.717894115, 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.717652037000001, 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.717598602999999, 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.717526423999999, 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:303 3.4s0 3:20577364:::smithi03327189-24:head [rmxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a in=40b,rmxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W in=40b,rmxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg in=36b,rmxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes in=40b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.703564866000001, 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011515044, 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 303 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532494+0000", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532496+0000", 2024-09-15T01:44:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8729999999999999e-06 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532507+0000", 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.13e-05 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532525+0000", 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7467000000000001e-05 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532548+0000", 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3320999999999999e-05 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536096+0000", 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003548556 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536222+0000", 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012545199999999999 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.538639+0000", 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024168390000000001 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543044+0000", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044051790000000004 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543940+0000", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00089649100000000002 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544009+0000", 2024-09-15T01:44:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8565999999999994e-05 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:498 3.4s0 3:25aaa82b:::smithi03327189-9:head [append 0~524288] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.632013932, 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011653834, 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 498 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604045+0000", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604049+0000", 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8739999999999997e-06 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604468+0000", 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041931899999999997 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604508+0000", 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9677e-05 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604524+0000", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6030000000000001e-05 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604637+0000", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113602 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.604816+0000", 2024-09-15T01:44:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.606494+0000", 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016776149999999999 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615523+0000", 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0090292470000000007 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615637+0000", 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011338200000000001 2024-09-15T01:44:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.615699+0000", 2024-09-15T01:44:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1970999999999993e-05 2024-09-15T01:44:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:511 3.4s0 3:20577364:::smithi03327189-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.600046393, 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014490421999999999, 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 511 2024-09-15T01:44:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636013+0000", 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:44:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636015+0000", 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2869999999999999e-06 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636020+0000", 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.34e-06 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636022+0000", 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3779999999999999e-06 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640680+0000", 2024-09-15T01:44:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046585400000000001 2024-09-15T01:44:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640820+0000", 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014002599999999999 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647530+0000", 2024-09-15T01:44:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067096409999999997 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647618+0000", 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7530999999999999e-05 2024-09-15T01:44:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.650503+0000", 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028851409999999999 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.6s1 58/53 ECSubWrite(tid=126, reqid=osd.0.0:126, at_version=58'32, trim_to=0'0, roll_forward_to=57'30))", 2024-09-15T01:44:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:51.342 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.667147848999999, 2024-09-15T01:44:51.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01167437, 2024-09-15T01:44:51.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.342 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568911+0000", 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000002e-07 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568942+0000", 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1117e-05 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568954+0000", 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1612e-05 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.568966+0000", 2024-09-15T01:44:51.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2051e-05 2024-09-15T01:44:51.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569058+0000", 2024-09-15T01:44:51.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2028999999999996e-05 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.569071+0000", 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2559e-05 2024-09-15T01:44:51.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580549+0000", 2024-09-15T01:44:51.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011478647999999999 2024-09-15T01:44:51.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.580585+0000", 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6050000000000002e-05 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7116757450000009, 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:44:51.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.355 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:51.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:44:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:44:51.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:44:51.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:51.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7067589230000006, 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:44:51.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:635 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7033046190000007, 2024-09-15T01:44:51.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013058536000000001, 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 635 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532754+0000", 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532757+0000", 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5770000000000001e-06 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532923+0000", 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016642800000000001 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532939+0000", 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5574e-05 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.532947+0000", 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0050000000000004e-06 2024-09-15T01:44:51.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536760+0000", 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038128760000000002 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536860+0000", 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100008 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538503+0000", 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001643451 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541264+0000", 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002760701 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545776+0000", 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045126050000000003 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545813+0000", 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6310999999999998e-05 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.375 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:639 3.8s1 3:1768942e:::smithi03327189-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6977612139999998, 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011868998, 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.376 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 639 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538298+0000", 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9799999999999999e-07 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538300+0000", 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8199999999999999e-06 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538306+0000", 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9689999999999999e-06 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.538309+0000", 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4659999999999999e-06 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.543951+0000", 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056413419999999997 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.544035+0000", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4798999999999999e-05 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.546810+0000", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027750880000000002 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.546900+0000", 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9216999999999996e-05 2024-09-15T01:44:51.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.550167+0000", 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032668990000000002 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.382 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:51.384 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:51.380+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:51.485 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:51.483+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:51.490 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:51.490 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:51.513 INFO:tasks.rados.rados.0.smithi033.stdout:278: done (6 left) 2024-09-15T01:44:51.513 INFO:tasks.rados.rados.0.smithi033.stdout:279: snap_remove snap 18 2024-09-15T01:44:51.541 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:44:51.586 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:51.579+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:51.608 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.608 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:51.609 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:51.609 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:51.609 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:51.686 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:51.679+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.192803065, 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.190317540999999, 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:51.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.182814289, 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:44:51.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:44:51.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.066682527999999, 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.062253918, 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:44:51.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:549 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.066804327, 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072147799999999996, 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 549 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645256+0000", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3900000000000005e-07 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645259+0000", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8839999999999998e-06 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645268+0000", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4149999999999999e-06 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645271+0000", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.841e-06 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645937+0000", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066678699999999996 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.646068+0000", 2024-09-15T01:44:51.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013108799999999999 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.652471+0000", 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064022259999999996 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.066761869, 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:44:51.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:44:51.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:44:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.961289793000001, 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:44:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:44:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.957850219000001, 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:44:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:44:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.953064277999999, 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:44:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.772 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1949066610000001, 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1948557409999996, 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:44:51.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:44:51.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:44:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:44:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:44:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:44:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:44:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1947907310000003, 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:44:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1828363370000003, 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088343680000000004, 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538058+0000", 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005438109 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.179618123, 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:44:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1755740570000004, 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.2s6 68/53 ECSubWrite(tid=191, reqid=osd.3.0:191, at_version=68'30, trim_to=0'0, roll_forward_to=64'28))", 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.526820+0000", 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.185240168, 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072512599999999998, 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526820+0000", 2024-09-15T01:44:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526820+0000", 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526821+0000", 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7400000000000003e-07 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.528550+0000", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001729092 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.528564+0000", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4474999999999999e-05 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.528572+0000", 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4610000000000002e-06 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.528661+0000", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9316999999999999e-05 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.528673+0000", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1863e-05 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.534033+0000", 2024-09-15T01:44:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053601719999999999 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.534071+0000", 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8306000000000002e-05 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/53 ECSubWrite(tid=124, reqid=client.4632.0:725, at_version=71'52, trim_to=0'0, roll_forward_to=68'50))", 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.157593627, 2024-09-15T01:44:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072912799999999998, 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5000000000000001e-07 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558747+0000", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042791749999999996 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558803+0000", 2024-09-15T01:44:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6045000000000003e-05 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558813+0000", 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558899+0000", 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5898999999999999e-05 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558908+0000", 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9570000000000008e-06 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561716+0000", 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028077710000000001 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561758+0000", 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2144000000000002e-05 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.18531587899999999, 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079190819999999992, 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526745+0000", 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5100000000000001e-07 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526765+0000", 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0860999999999999e-05 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526775+0000", 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9599999999999995e-06 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526782+0000", 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.426e-06 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528938+0000", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021560450000000001 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528947+0000", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4499999999999993e-06 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534617+0000", 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056700489999999999 2024-09-15T01:44:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534663+0000", 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.604e-05 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.183622224, 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079955070000000007, 2024-09-15T01:44:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528469+0000", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0893999999999999e-05 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528482+0000", 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2745000000000001e-05 2024-09-15T01:44:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528489+0000", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8920000000000003e-06 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529953+0000", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001463696 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529965+0000", 2024-09-15T01:44:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2004999999999999e-05 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536411+0000", 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064466819999999996 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536433+0000", 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2109000000000001e-05 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:51.823 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:51.825 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:51.819+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:51.925 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:51.923+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:51.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:51.926 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:51.926 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:51.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:52.025 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:52.019+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:52.038 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:52.038 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:52.038 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:52.038 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:52.039 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:52.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:52.126 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:52.123+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.638035874000003, 2024-09-15T01:44:52.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.637981635000003, 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:52.160 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.637962496999997, 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:52.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.628987596999998, 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:52.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:52.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.628919934999999, 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.175 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.628894194000001, 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:52.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.615823384999999, 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs1 43/34 ECSubWrite(tid=109, reqid=osd.5.0:109, at_version=43'21, trim_to=0'0, roll_forward_to=41'19))", 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.614781480000001, 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010233333000000001, 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532653+0000", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999998e-07 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532674+0000", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0486999999999999e-05 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532692+0000", 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8309e-05 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532698+0000", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4819999999999999e-06 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532804+0000", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010524899999999999 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532813+0000", 2024-09-15T01:44:52.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8940000000000005e-06 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542856+0000", 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010042005 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542886+0000", 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0574000000000002e-05 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.3s1 43/34 ECSubWrite(tid=14, reqid=client.4632.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=43'6))", 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.613960675000001, 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093499180000000005, 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533474+0000", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0499999999999999e-07 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533484+0000", 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7000000000000003e-06 2024-09-15T01:44:52.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533495+0000", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1260999999999999e-05 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533501+0000", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533664+0000", 2024-09-15T01:44:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016271499999999999 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.533674+0000", 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7520000000000008e-06 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542794+0000", 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091202669999999996 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542823+0000", 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9498000000000001e-05 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:481 3.9s1 3:9e656ec5:::smithi03327189-15:head [append 0~475136] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.611997464, 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095211319999999999, 2024-09-15T01:44:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 481 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535437+0000", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535439+0000", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5320000000000001e-06 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535767+0000", 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032738100000000002 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535789+0000", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1917e-05 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535882+0000", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3171000000000006e-05 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.535944+0000", 2024-09-15T01:44:52.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2124000000000004e-05 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.536068+0000", 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012412600000000001 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.537727+0000", 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016587010000000001 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544825+0000", 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070983599999999997 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544893+0000", 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8188999999999994e-05 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.544958+0000", 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4630999999999998e-05 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.523624269999999, 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:44:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:44:52.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:44:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:537 3.bs0 3:dc585df2:::smithi03327189-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.523558194, 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011082247999999999, 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 537 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.206 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623877+0000", 2024-09-15T01:44:52.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623881+0000", 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623895+0000", 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3091000000000001e-05 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623902+0000", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.153e-06 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624357+0000", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045563899999999998 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624496+0000", 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013903799999999999 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634734+0000", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01023717 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634849+0000", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001154 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634958+0000", 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109345 2024-09-15T01:44:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:555 3.bs0 3:d32e3b1b:::smithi03327189-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.487261628000001, 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094173030000000001, 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 555 2024-09-15T01:44:52.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660173+0000", 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660174+0000", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660177+0000", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7069999999999998e-06 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660199+0000", 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1659999999999999e-05 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.660205+0000", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7869999999999998e-06 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663351+0000", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031458609999999998 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.663435+0000", 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3947000000000001e-05 2024-09-15T01:44:52.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668442+0000", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050072629999999996 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.668572+0000", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013023000000000001 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.669590+0000", 2024-09-15T01:44:52.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010177529999999999 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5443156590000005, 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:44:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:52.222 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5441821939999993, 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:52.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:44:52.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:44:52.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.62349091400000001, 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:44:52.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.61985948300000004, 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:52.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:44:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:52.237 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.61978580800000005, 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:44:52.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:44:52.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 72/53 ECSubWriteReply(tid=206, last_complete=72'40, committed=1, applied=1))", 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527775+0000", 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.61965963999999996, 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093303180000000006, 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527775+0000", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527775+0000", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527775+0000", 2024-09-15T01:44:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0999999999999999e-07 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527778+0000", 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2590000000000001e-06 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527787+0000", 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7099999999999996e-06 2024-09-15T01:44:52.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527793+0000", 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-06 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.537084+0000", 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092905769999999995 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.537105+0000", 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1002e-05 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 72/53 ECSubWrite(tid=159, reqid=osd.0.0:159, at_version=72'62, trim_to=0'0, roll_forward_to=68'60))", 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.619571753, 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011041109, 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527863+0000", 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.96e-07 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527880+0000", 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7333000000000002e-05 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527889+0000", 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.568e-06 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527894+0000", 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3850000000000001e-06 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528016+0000", 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121822 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528025+0000", 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0350000000000007e-06 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538848+0000", 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010822976 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538904+0000", 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5594000000000003e-05 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:52.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:52.251 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:52.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:52.253 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:52.248+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:52.353 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:52.347+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:52.362 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:52.363 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:52.363 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:52.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:52.454 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:52.447+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:52.475 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:52.475 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:52.475 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:52.475 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:52.475 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:52.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:279: done (6 left) 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:280: rollback oid 37 current snap is 21 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 37 to 19 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:281: copy_from oid 24 from oid 45 current snap is 21 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 47 current snap is 21 2024-09-15T01:44:52.521 INFO:tasks.rados.rados.0.smithi033.stdout:282: seq_num 129 ranges {1458176=622592,2080768=655360,2736128=688128,3424256=212992} 2024-09-15T01:44:52.533 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:44:52.543 INFO:tasks.rados.rados.0.smithi033.stdout:282: writing smithi03327189-47 from 1458176 to 2080768 tid 1 2024-09-15T01:44:52.549 INFO:tasks.rados.rados.0.smithi033.stdout:282: writing smithi03327189-47 from 2080768 to 2736128 tid 2 2024-09-15T01:44:52.554 INFO:tasks.rados.rados.0.smithi033.stdout:282: writing smithi03327189-47 from 2736128 to 3424256 tid 3 2024-09-15T01:44:52.554 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:52.547+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:282: writing smithi03327189-47 from 3424256 to 3637248 tid 4 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:283: read oid 30 snap 19 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:283: expect (ObjNum -1804580496 snap 21947 seq_num 2490736192) 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:284: rmattr oid 38 current snap is 21 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 27 v 13 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:276: finishing rollback tid 0 to smithi03327189-15 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:276: finishing rollback tid 1 to smithi03327189-15 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:275: done (10 left) 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:285: rmattr oid 8 current snap is 21 2024-09-15T01:44:52.556 INFO:tasks.rados.rados.0.smithi033.stdout:285: done (10 left) 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:286: delete oid 34 current snap is 21 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:276: finishing rollback tid 2 to smithi03327189-15 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 28 (ObjNum 97 snap 13 seq_num 97) dirty exists 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:272: finishing copy_from racing read to smithi03327189-20 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 33 v 36 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:44:52.557 INFO:tasks.rados.rados.0.smithi033.stdout:277: expect (ObjNum 93 snap 11 seq_num 93) 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.076650401999999, 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:44:52.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:44:52.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:44:52.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:44:52.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:52.606 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.076592056000003, 2024-09-15T01:44:52.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:44:52.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:44:52.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:52.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.076513298999998, 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:44:52.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:44:52.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.076333298000002, 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 16.066023343000001, 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:44:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:44:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.010777597000001, 2024-09-15T01:44:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.632 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:44:52.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:44:52.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.001935528000001, 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.637 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:44:52.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:44:52.641 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.971055907, 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.642 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:52.643 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:44:52.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.647 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.970976539, 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:44:52.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:44:52.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:44:52.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:52.653 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.951086158000001, 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.654 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:44:52.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:44:52.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.949723496000001, 2024-09-15T01:44:52.659 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:44:52.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:44:52.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.949708971, 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:44:52.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:547 3.0s0 3:08c156cd:::smithi03327189-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.949660549000001, 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0091027969999999993, 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 547 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645643+0000", 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.79e-07 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645644+0000", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.699e-06 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645650+0000", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.942e-06 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645653+0000", 2024-09-15T01:44:52.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.452e-06 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653028+0000", 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073755519999999996 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653094+0000", 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5718999999999999e-05 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654745+0000", 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016511539999999999 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.927323210000001, 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.675 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:44:52.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.679 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.680 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:560 3.0s0 3:0899e339:::smithi03327189-38:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.923590761, 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090281700000000003, 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 560 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.683 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671713+0000", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671717+0000", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1749999999999998e-06 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671737+0000", 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.948e-05 2024-09-15T01:44:52.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671744+0000", 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9569999999999996e-06 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676355+0000", 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046115949999999996 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.685 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676528+0000", 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017328999999999999 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677261+0000", 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073246900000000002 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677982+0000", 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072084199999999997 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.686 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680660+0000", 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002678541 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680740+0000", 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9963000000000007e-05 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.687 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.923552888, 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.688 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:44:52.689 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.690 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:44:52.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.692 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.078428046, 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.693 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:52.694 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.695 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:44:52.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:44:52.698 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:675 3.cs0 3:3584cd82:::smithi03327189-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:52.699 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.0783110059999998, 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017344149999999999, 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 675 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516993+0000", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5600000000000001e-06 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517001+0000", 2024-09-15T01:44:52.701 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517020+0000", 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9364999999999998e-05 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517028+0000", 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6229999999999997e-06 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.702 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517199+0000", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017127399999999999 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7309999999999998e-06 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524428+0000", 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072231680000000003 2024-09-15T01:44:52.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524522+0000", 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4467000000000002e-05 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534164+0000", 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0096419079999999994 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.704 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534243+0000", 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9296999999999997e-05 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534336+0000", 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2608000000000005e-05 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:680 3.0s0 3:0a6b0087:::smithi03327189-4:head [append 0~524288] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:52.705 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.0521927130000002, 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010598392999999999, 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 680 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543113+0000", 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5490000000000001e-06 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.707 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544026+0000", 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091366600000000004 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544053+0000", 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7015000000000001e-05 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544068+0000", 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4977000000000001e-05 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544122+0000", 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3804000000000002e-05 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544197+0000", 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5139e-05 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:52.709 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.545372+0000", 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001175024 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.547186+0000", 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001814026 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553517+0000", 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063304659999999999 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.710 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553709+0000", 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019172699999999999 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:685 3.0s0 3:0a6b0087:::smithi03327189-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.0385470479999999, 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01007038, 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:52.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 685 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:52.712 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8299999999999997e-07 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556760+0000", 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556766+0000", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556770+0000", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6109999999999998e-06 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.562875+0000", 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061050740000000003 2024-09-15T01:44:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.563061+0000", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000186066 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564501+0000", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014400400000000001 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564606+0000", 2024-09-15T01:44:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010522399999999999 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.566826+0000", 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002219782 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:52.716 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:52.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:52.719 INFO:tasks.rados.rados.0.smithi033.stdout:272: finishing copy_from to smithi03327189-20 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 30 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:273: finishing copy_from racing read to smithi03327189-35 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:273: finishing copy_from to smithi03327189-35 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 35 v 38 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:281: finishing copy_from to smithi03327189-24 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:281: got expected ENOENT (src dne) 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:281: finishing copy_from racing read to smithi03327189-24 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:280: finishing rollback tid 0 to smithi03327189-37 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:280: finishing rollback tid 1 to smithi03327189-37 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:272: done (10 left) 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:273: done (9 left) 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:274: done (8 left) 2024-09-15T01:44:52.720 INFO:tasks.rados.rados.0.smithi033.stdout:276: done (7 left) 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:277: done (6 left) 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:281: done (5 left) 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:286: done (4 left) 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:287: copy_from oid 11 from oid 49 current snap is 21 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:280: finishing rollback tid 2 to smithi03327189-37 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 37 v 22 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 1 to smithi03327189-47 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 0 (ObjNum 108 snap 13 seq_num 108) dirty exists 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:283: expect (ObjNum -1804580496 snap 21947 seq_num 2490736192) 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 2 to smithi03327189-47 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 3 to smithi03327189-47 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 4 to smithi03327189-47 2024-09-15T01:44:52.721 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 5 to smithi03327189-47 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:282: finishing write tid 6 to smithi03327189-47 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 125 (ObjNum 129 snap 21 seq_num 129) dirty exists 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:282: left oid 47 (ObjNum 129 snap 21 seq_num 129) 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:280: done (4 left) 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:282: done (3 left) 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:283: done (2 left) 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:284: done (1 left) 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:288: snap_create 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:287: finishing copy_from racing read to smithi03327189-11 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:287: finishing copy_from to smithi03327189-11 2024-09-15T01:44:52.722 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 11 v 13 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:44:52.722 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:52.712+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:52.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:52.811+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:52.830 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:52.830 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:52.830 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:52.831 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:52.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:52.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:52.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:52.860 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-15T01:44:52.860 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T01:44:52.860 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T01:44:52.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-15T01:44:52.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:52.919 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:52.911+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:52.969 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:52.970 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T01:44:52.970 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:52.985 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:52.986 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:52.986 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:52.986 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:52.986 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:52.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:53.021 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:53.015+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:53.035 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:53.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:53.078 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T01:44:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 56.698453569000002, 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006287684, 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 5 2024-09-15T01:44:53.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394852+0000", 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7460000000000001e-06 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394870+0000", 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8074e-05 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394905+0000", 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4876000000000003e-05 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394923+0000", 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7691000000000001e-05 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395069+0000", 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014603000000000001 2024-09-15T01:44:53.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395085+0000", 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6143000000000001e-05 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398289+0000", 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003203759 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.103 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398489+0000", 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000200223 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398670+0000", 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000180359 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.400465+0000", 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001795718 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401043+0000", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057742000000000004 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401080+0000", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7283000000000002e-05 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401137+0000", 2024-09-15T01:44:53.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7361999999999997e-05 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 55.683556158999998, 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0068396289999999998, 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.106 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 7 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409750+0000", 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0699999999999998e-06 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409763+0000", 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.325e-05 2024-09-15T01:44:53.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409805+0000", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1412000000000003e-05 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409839+0000", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3692999999999997e-05 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410004+0000", 2024-09-15T01:44:53.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016576300000000001 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410032+0000", 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8076e-05 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413650+0000", 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036180219999999998 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413889+0000", 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00023881400000000001 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.414266+0000", 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00037692599999999998 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.415749+0000", 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001483301 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416329+0000", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057982400000000005 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416478+0000", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014846299999999999 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416587+0000", 2024-09-15T01:44:53.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109015 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:222 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.575043823999998, 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026140312999999998, 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.113 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 222 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518262+0000", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1090000000000001e-06 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518270+0000", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518286+0000", 2024-09-15T01:44:53.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6081000000000001e-05 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518298+0000", 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1941e-05 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518644+0000", 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034667699999999999 2024-09-15T01:44:53.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518650+0000", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526775+0000", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081258210000000001 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526967+0000", 2024-09-15T01:44:53.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191991 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.544400+0000", 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017432277999999999 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:223 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.574981911999998, 2024-09-15T01:44:53.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027026921999999998, 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 223 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518322+0000", 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518328+0000", 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4210000000000003e-06 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518338+0000", 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.022e-05 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518343+0000", 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.66e-06 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518713+0000", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036931399999999998 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518715+0000", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.841e-06 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527539+0000", 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0088243579999999992 2024-09-15T01:44:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527752+0000", 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000212888 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017596414000000001 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:224 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.57494896, 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028715145000000001, 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 224 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518355+0000", 2024-09-15T01:44:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518359+0000", 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9550000000000002e-06 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518367+0000", 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518372+0000", 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8420000000000004e-06 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518770+0000", 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039736300000000002 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518772+0000", 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2019999999999998e-06 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528208+0000", 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094362790000000005 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528403+0000", 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019513399999999999 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.547069+0000", 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018666242999999999 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.9s4 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.565752209999999, 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0062097970000000004, 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527552+0000", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5199999999999996e-07 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528961+0000", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014095609999999999 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528973+0000", 2024-09-15T01:44:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1335000000000001e-05 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528990+0000", 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7359000000000001e-05 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529050+0000", 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0235000000000002e-05 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529062+0000", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1531e-05 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533731+0000", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046689110000000004 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533761+0000", 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0113000000000001e-05 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/32 ECSubWriteReply(tid=61, last_complete=33'22, committed=1, applied=1))", 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 40.565306405999998, 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050195400000000003, 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527998+0000", 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0839999999999999e-06 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528005+0000", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7209999999999997e-06 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528018+0000", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3818000000000001e-05 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528026+0000", 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1359999999999999e-06 2024-09-15T01:44:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533000+0000", 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049747359999999996 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533016+0000", 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6045e-05 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:255 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 34.572647361000001, 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049454870000000001, 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 255 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520660+0000", 2024-09-15T01:44:53.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6600000000000001e-06 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520673+0000", 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3406e-05 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520707+0000", 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3902999999999997e-05 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520727+0000", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0588999999999999e-05 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520839+0000", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011144199999999999 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.521184+0000", 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034537000000000001 2024-09-15T01:44:53.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.525601+0000", 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044171169999999999 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:256 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 34.572528656999999, 2024-09-15T01:44:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061411549999999997, 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 256 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520776+0000", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5e-06 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520786+0000", 2024-09-15T01:44:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520807+0000", 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1169999999999999e-05 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520819+0000", 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1589e-05 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522420+0000", 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001600953 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522494+0000", 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4438999999999997e-05 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.526916+0000", 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044215110000000004 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:257 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 34.572468561999997, 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006854008, 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 257 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.147 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520836+0000", 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520844+0000", 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8599999999999993e-06 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520862+0000", 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7859999999999998e-05 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520872+0000", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522823+0000", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019506020000000001 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522947+0000", 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012457899999999999 2024-09-15T01:44:53.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.527689+0000", 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004741398 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(osd.1.17:1 3.0s0 3:08c156cd:::smithi03327189-34:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.567375263999999, 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012067648, 2024-09-15T01:44:53.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.152 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525932+0000", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.693e-06 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525943+0000", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0917000000000001e-05 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525964+0000", 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0993e-05 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525975+0000", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1823e-05 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527736+0000", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001760345 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527967+0000", 2024-09-15T01:44:53.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230769 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.537996+0000", 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010029108 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=50, reqid=client.4632.0:290, at_version=43'17, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.560111296999999, 2024-09-15T01:44:53.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061827899999999996, 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:53.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.68e-07 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533416+0000", 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000223594 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533428+0000", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649e-05 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.534981+0000", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001553029 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535038+0000", 2024-09-15T01:44:53.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6560000000000001e-05 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535050+0000", 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1885999999999999e-05 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539340+0000", 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004289836 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539375+0000", 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5067999999999997e-05 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=51, reqid=client.4632.0:291, at_version=43'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.558270123, 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049865860000000003, 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:53.160 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:53.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9200000000000002e-07 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535160+0000", 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126678 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535171+0000", 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0783e-05 2024-09-15T01:44:53.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535178+0000", 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9809999999999997e-06 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535795+0000", 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061714799999999998 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535804+0000", 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0680000000000003e-06 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539989+0000", 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041847079999999997 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.540020+0000", 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0927999999999999e-05 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:393 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 0] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 26.586141596000001, 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077679840000000003, 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 393 2024-09-15T01:44:53.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507165+0000", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2729999999999998e-06 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507179+0000", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4344e-05 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507215+0000", 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5509000000000002e-05 2024-09-15T01:44:53.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507230+0000", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5116000000000001e-05 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507374+0000", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014417100000000001 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507400+0000", 2024-09-15T01:44:53.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6222000000000001e-05 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511249+0000", 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038487730000000002 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511443+0000", 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194234 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.169 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.514930+0000", 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034863419999999999 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:394 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 26.586048357999999, 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081504530000000002, 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 394 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507256+0000", 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1230000000000001e-06 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507260+0000", 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3970000000000004e-06 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507278+0000", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8077e-05 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507285+0000", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507540+0000", 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000254541 2024-09-15T01:44:53.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507553+0000", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2962999999999999e-05 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511752+0000", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041991509999999999 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511891+0000", 2024-09-15T01:44:53.174 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013930299999999999 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515075+0000", 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0031832549999999999 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515236+0000", 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000161096 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.175 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515405+0000", 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016956100000000001 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:396 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~671744] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.576774667999999, 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014871097, 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 396 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.177 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516531+0000", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8700000000000001e-06 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517417+0000", 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00088526499999999999 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517445+0000", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8351e-05 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517458+0000", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2894000000000001e-05 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517618+0000", 2024-09-15T01:44:53.179 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000160025 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517871+0000", 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025296800000000001 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.520096+0000", 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022246259999999999 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.180 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523463+0000", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0033674690000000001 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531316+0000", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078531639999999993 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531400+0000", 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3465000000000001e-05 2024-09-15T01:44:53.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:397 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~770048] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.569427212000001, 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0082945069999999996, 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 397 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523879+0000", 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2610000000000002e-06 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525106+0000", 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266060000000001 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.184 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525135+0000", 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525148+0000", 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3892e-05 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525342+0000", 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019338100000000001 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.185 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525610+0000", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026807099999999999 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.528197+0000", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025871959999999999 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531425+0000", 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032278430000000002 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532119+0000", 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069357900000000005 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532171+0000", 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2008000000000001e-05 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.187 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:398 3.5s0 3:ac2d3331:::smithi03327189-18:head [setxattr _header (57) in=7b] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.568133831000001, 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073765130000000003, 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 398 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525171+0000", 2024-09-15T01:44:53.189 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525178+0000", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1239999999999999e-06 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525193+0000", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.535e-05 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525203+0000", 2024-09-15T01:44:53.190 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4749999999999999e-06 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530018+0000", 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048148799999999997 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530132+0000", 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000114434 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530708+0000", 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057595400000000005 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531935+0000", 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266790000000001 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532465+0000", 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00052987900000000003 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532546+0000", 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1151999999999998e-05 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:399 3.5s0 3:ac2d3331:::smithi03327189-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.567779360999999, 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090256980000000004, 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 399 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.194 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525525+0000", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5009999999999999e-06 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525531+0000", 2024-09-15T01:44:53.195 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1120000000000001e-06 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525550+0000", 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8819000000000001e-05 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525559+0000", 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.833e-06 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.196 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531733+0000", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061738590000000003 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531869+0000", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013559600000000001 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532982+0000", 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011133320000000001 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.533176+0000", 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019444500000000001 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.534550+0000", 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013732009999999999 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 48/34 ECSubWriteReply(tid=157, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.567705400000001, 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005543699, 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:53.199 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525599+0000", 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525606+0000", 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7349999999999999e-06 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525621+0000", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5559000000000001e-05 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525628+0000", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6560000000000003e-06 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531120+0000", 2024-09-15T01:44:53.201 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0054919829999999998 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531142+0000", 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1671e-05 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:53.202 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:53.204 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:53.204 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:53.204 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:53.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:53.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:53.205 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:53.200+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:53.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:53.211 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:53.235 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:53.259 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:53.283 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:53.286 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:53.304 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:53.295+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:53.317 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.318 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:53.405 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:53.403+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:53.429 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.429 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:53.505 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:53.499+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:53.520 INFO:tasks.rados.rados.0.smithi033.stdout:287: done (1 left) 2024-09-15T01:44:53.520 INFO:tasks.rados.rados.0.smithi033.stdout:288: done (0 left) 2024-09-15T01:44:53.520 INFO:tasks.rados.rados.0.smithi033.stdout:289: copy_from oid 4 from oid 11 current snap is 22 2024-09-15T01:44:53.520 INFO:tasks.rados.rados.0.smithi033.stdout:290: snap_remove snap 20 2024-09-15T01:44:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.547 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:53.606 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:53.599+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:53.649 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.649 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:53.708 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:53.699+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:53.757 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.757 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.766 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:53.809 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:53.804+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:53.864 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.864 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.873 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:53.920 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:53.915+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:53.983 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:53.983 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:53.992 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:54.010 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:54.004+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:54.094 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.094 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:54.111 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:54.104+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:54.204 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.204 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.212 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:54.204+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:54.213 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:54.313 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:54.307+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:54.313 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.313 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:54.414 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:54.407+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:54.420 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.420 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:54.514 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:54.508+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:54.526 INFO:tasks.rados.rados.0.smithi033.stdout:290: done (1 left) 2024-09-15T01:44:54.526 INFO:tasks.rados.rados.0.smithi033.stdout:291: snap_remove snap 19 2024-09-15T01:44:54.547 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.548 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:44:54.557 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:54.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:54.608+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:54.648 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.648 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.657 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:54.716 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:54.711+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:54.758 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.758 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.768 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:54.817 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:54.812+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:54.868 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.868 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.877 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:54.918 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:54.911+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:54.985 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:55.019 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:55.015+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:55.120 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:55.115+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:55.129 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:55.129 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:55.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:44:55.221 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:55.215+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:55.242 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:55.242 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:55.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:44:55.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:44:55.321 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:55.315+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:55.364 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:55.364 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:55.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:44:55.422 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:55.415+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:55.437 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:55.456 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:55.477 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:55.480 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:55.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:44:55.503 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:55.523 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:55.515+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:55.530 INFO:tasks.rados.rados.0.smithi033.stdout:291: done (1 left) 2024-09-15T01:44:55.530 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 15 current snap is 22 2024-09-15T01:44:55.530 INFO:tasks.rados.rados.0.smithi033.stdout:292: seq_num 131 ranges {2113536=655360,2768896=425984,3194880=638976,3833856=491520} 2024-09-15T01:44:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:44:55.548 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:55.556 INFO:tasks.rados.rados.0.smithi033.stdout:292: writing smithi03327189-15 from 2113536 to 2768896 tid 1 2024-09-15T01:44:55.559 INFO:tasks.rados.rados.0.smithi033.stdout:292: writing smithi03327189-15 from 2768896 to 3194880 tid 2 2024-09-15T01:44:55.564 INFO:tasks.rados.rados.0.smithi033.stdout:292: writing smithi03327189-15 from 3194880 to 3833856 tid 3 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:292: writing smithi03327189-15 from 3833856 to 4325376 tid 4 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:289: finishing copy_from racing read to smithi03327189-4 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:289: finishing copy_from to smithi03327189-4 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 13 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 1 to smithi03327189-15 2024-09-15T01:44:55.568 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 2 to smithi03327189-15 2024-09-15T01:44:55.569 INFO:tasks.rados.rados.0.smithi033.stdout:289: done (1 left) 2024-09-15T01:44:55.569 INFO:tasks.rados.rados.0.smithi033.stdout:293: copy_from oid 2 from oid 18 current snap is 22 2024-09-15T01:44:55.569 INFO:tasks.rados.rados.0.smithi033.stdout:294: rmattr oid 3 current snap is 22 2024-09-15T01:44:55.569 INFO:tasks.rados.rados.0.smithi033.stdout:294: done (2 left) 2024-09-15T01:44:55.569 INFO:tasks.rados.rados.0.smithi033.stdout:295: snap_create 2024-09-15T01:44:55.570 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 3 to smithi03327189-15 2024-09-15T01:44:55.574 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 4 to smithi03327189-15 2024-09-15T01:44:55.574 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 5 to smithi03327189-15 2024-09-15T01:44:55.577 INFO:tasks.rados.rados.0.smithi033.stdout:292: finishing write tid 6 to smithi03327189-15 2024-09-15T01:44:55.577 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 15 v 42 (ObjNum 131 snap 22 seq_num 131) dirty exists 2024-09-15T01:44:55.577 INFO:tasks.rados.rados.0.smithi033.stdout:292: left oid 15 (ObjNum 131 snap 22 seq_num 131) 2024-09-15T01:44:55.586 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:55.598 INFO:tasks.rados.rados.0.smithi033.stdout:293: finishing copy_from racing read to smithi03327189-2 2024-09-15T01:44:55.603 INFO:tasks.rados.rados.0.smithi033.stdout:293: finishing copy_from to smithi03327189-2 2024-09-15T01:44:55.603 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 2 v 85 (ObjNum 98 snap 13 seq_num 98) dirty exists 2024-09-15T01:44:55.609 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:55.609 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:55.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.079113986000003, 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:44:55.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:44:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:44:55.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:55.616 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.076682587999997, 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:44:55.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:44:55.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.076443773000001, 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:44:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:44:55.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.076412390000002, 2024-09-15T01:44:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:44:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:44:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:44:55.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.097204959999999, 2024-09-15T01:44:55.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.639 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:44:55.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:44:55.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.089290558000002, 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:44:55.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:44:55.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.089261295, 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:44:55.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:44:55.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:44:55.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.089685975999998, 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:44:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:44:55.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:44:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:44:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.087447515000001, 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:44:55.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:44:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:44:55.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.086877846, 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:44:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:44:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.086818878000003, 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:44:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:44:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.098920669999998, 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:44:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.098826892000002, 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:44:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:44:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.032182777999999, 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:44:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:44:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=117, reqid=osd.4.0:117, at_version=68'74, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.0910915009999993, 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011721416, 2024-09-15T01:44:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514471+0000", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514541+0000", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0955000000000001e-05 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514567+0000", 2024-09-15T01:44:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.597e-05 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514584+0000", 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6463000000000001e-05 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514727+0000", 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014272899999999999 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514754+0000", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7290000000000001e-05 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526159+0000", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011404621 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526191+0000", 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2840999999999999e-05 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/53 ECSubWrite(tid=118, reqid=osd.4.0:118, at_version=68'50, trim_to=0'0, roll_forward_to=67'48))", 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.0906842520000009, 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012184878999999999, 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514879+0000", 2024-09-15T01:44:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514938+0000", 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9142999999999999e-05 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514963+0000", 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5165999999999999e-05 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515032+0000", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8602999999999994e-05 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515160+0000", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012877200000000001 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515188+0000", 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7313000000000001e-05 2024-09-15T01:44:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527007+0000", 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011819079999999999 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527062+0000", 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5328999999999998e-05 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=68'76, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.088629869, 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014580965, 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516933+0000", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6249999999999999e-06 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516977+0000", 2024-09-15T01:44:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3551999999999999e-05 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517002+0000", 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4762000000000001e-05 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517016+0000", 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.401e-05 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517789+0000", 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077353199999999995 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517813+0000", 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3748e-05 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531475+0000", 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013662636000000001 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531513+0000", 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7100000000000001e-05 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:4 3.1s0 3:85bd41e4:::smithi03327189-14:head [assert-version v38,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0812825999999998, 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010562561999999999, 2024-09-15T01:44:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524279+0000", 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524282+0000", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524293+0000", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1588e-05 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524323+0000", 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9383e-05 2024-09-15T01:44:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524339+0000", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6314e-05 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524415+0000", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5636000000000006e-05 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.524530+0000", 2024-09-15T01:44:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115263 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.534841+0000", 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010311512 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:753 3.1s0 3:86165cc9:::smithi03327189-11:head [copy-from ver 13] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:44:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.0042098890000002, 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022114891000000001, 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 753 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:44:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601354+0000", 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6000000000000001e-06 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601366+0000", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601396+0000", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9901999999999999e-05 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601406+0000", 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601519+0000", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112546 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601677+0000", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015800100000000001 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.614051+0000", 2024-09-15T01:44:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012374232000000001 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617994+0000", 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039432089999999996 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623415+0000", 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054204240000000001 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623466+0000", 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1972000000000003e-05 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:754 3.1s0 3:86165cc9:::smithi03327189-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.004130242, 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015926781000000001, 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 754 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601433+0000", 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4270000000000001e-06 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601437+0000", 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2899999999999996e-06 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601452+0000", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4939999999999999e-05 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601461+0000", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601979+0000", 2024-09-15T01:44:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051790899999999999 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.602057+0000", 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8237000000000003e-05 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617204+0000", 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015147361999999999 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617280+0000", 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6318999999999999e-05 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617358+0000", 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7570000000000004e-05 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:55.739 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:55.740 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:55.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:44:55.742 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:55.742 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:55.742 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:55.742 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:55.743 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:55.743 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:55.744 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:55.736+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:55.842 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:55.839+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:55.893 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:55.893 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:55.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:44:55.943 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:55.939+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:56.004 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.004 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:56.004 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:56.004 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:56.004 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:44:56.045 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:56.039+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.590717142000003, 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:44:56.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.590686198, 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.122 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:44:56.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.126 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.593806377, 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:44:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.133 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.593736845000002, 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:44:56.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:44:56.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.593714308999999, 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:44:56.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:44:56.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.593647117000003, 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:44:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:44:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.593577592000003, 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:44:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:44:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:44:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.593335514000003, 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:44:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:44:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.593282080000002, 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:44:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:44:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.593209901000002, 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:44:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:511 3.4s0 3:20577364:::smithi03327189-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.475729869999999, 2024-09-15T01:44:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014490421999999999, 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 511 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636013+0000", 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636015+0000", 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2869999999999999e-06 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636020+0000", 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.34e-06 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636022+0000", 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3779999999999999e-06 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640680+0000", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046585400000000001 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640820+0000", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014002599999999999 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647530+0000", 2024-09-15T01:44:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067096409999999997 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647618+0000", 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7530999999999999e-05 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.650503+0000", 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028851409999999999 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.587359222, 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:44:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.5824424, 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:44:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:44:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 73/53 ECSubWrite(tid=132, reqid=osd.4.0:132, at_version=73'84, trim_to=0'0, roll_forward_to=72'82))", 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5948564310000002, 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015282341, 2024-09-15T01:44:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516946+0000", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9145999999999999e-05 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516976+0000", 2024-09-15T01:44:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0236e-05 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516993+0000", 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517168+0000", 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017460800000000001 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517197+0000", 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8813e-05 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532115+0000", 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014917629 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532168+0000", 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3596e-05 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:740 3.4s0 3:20577364:::smithi03327189-24:head [copy-from ver 0] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5921731289999999, 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016305848000000001, 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 740 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519573+0000", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-06 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519590+0000", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519639+0000", 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9432999999999999e-05 2024-09-15T01:44:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519658+0000", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9083e-05 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521437+0000", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017789539999999999 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521452+0000", 2024-09-15T01:44:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4952e-05 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530454+0000", 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.009001963 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530578+0000", 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012395599999999999 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535875+0000", 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052969169999999999 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:741 3.4s0 3:20577364:::smithi03327189-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5919958410000001, 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016411578999999999, 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 741 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:44:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519748+0000", 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.04e-06 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519758+0000", 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5929999999999995e-06 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519795+0000", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6860000000000003e-05 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519810+0000", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5082999999999999e-05 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521558+0000", 2024-09-15T01:44:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017476 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521568+0000", 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0344e-05 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530788+0000", 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092198539999999995 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530875+0000", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7698e-05 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535967+0000", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918409999999997 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536066+0000", 2024-09-15T01:44:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8281000000000003e-05 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536158+0000", 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2385000000000004e-05 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=183, reqid=client.4632.0:738, at_version=73'21, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:44:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.588202398, 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018357772000000001, 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523541+0000", 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.24e-06 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523599+0000", 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7500999999999997e-05 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523628+0000", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9193999999999999e-05 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523640+0000", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2276000000000001e-05 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531608+0000", 2024-09-15T01:44:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079682079999999992 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531619+0000", 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0293e-05 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541866+0000", 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010247114999999999 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541898+0000", 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1945e-05 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 73/34 ECSubWrite(tid=58, reqid=osd.2.0:58, at_version=73'43, trim_to=0'0, roll_forward_to=72'41))", 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5854630749999998, 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015869550999999999, 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526280+0000", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999996e-07 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526306+0000", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6282e-05 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526322+0000", 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5977999999999998e-05 2024-09-15T01:44:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526330+0000", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.191e-06 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526451+0000", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120526 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526459+0000", 2024-09-15T01:44:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542088+0000", 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015629295000000001 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542149+0000", 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0247000000000001e-05 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=184, reqid=client.4632.0:739, at_version=73'22, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.584330493, 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014513999, 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527413+0000", 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9100000000000002e-07 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527437+0000", 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3961e-05 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527451+0000", 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3966e-05 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527464+0000", 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3030999999999999e-05 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.533825+0000", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063618429999999998 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.533837+0000", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1871999999999999e-05 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541902+0000", 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.008064903 2024-09-15T01:44:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541926+0000", 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3532000000000001e-05 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5842696319999998, 2024-09-15T01:44:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014773232000000001, 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527518+0000", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4802000000000002e-05 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527528+0000", 2024-09-15T01:44:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8640000000000008e-06 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527534+0000", 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7590000000000003e-06 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534080+0000", 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065466159999999999 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534092+0000", 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1953999999999999e-05 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542223+0000", 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081309510000000008 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542246+0000", 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2776999999999999e-05 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.230 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:44:56.232 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:56.223+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:56.332 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:56.323+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:56.337 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.337 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:56.337 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:56.337 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:44:56.433 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:56.431+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:56.447 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.447 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:56.447 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:56.447 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:56.447 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:44:56.518 INFO:tasks.rados.rados.0.smithi033.stdout:292: done (2 left) 2024-09-15T01:44:56.518 INFO:tasks.rados.rados.0.smithi033.stdout:293: done (1 left) 2024-09-15T01:44:56.518 INFO:tasks.rados.rados.0.smithi033.stdout:295: done (0 left) 2024-09-15T01:44:56.518 INFO:tasks.rados.rados.0.smithi033.stdout:296: snap_remove snap 22 2024-09-15T01:44:56.533 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:56.527+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:56.564 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.564 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:56.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.040726112000002, 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.565 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:44:56.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.038240588000001, 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:44:56.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:44:56.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.030737336000001, 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:44:56.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:44:56.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.914605575, 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:44:56.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.586 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.910176965000002, 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:44:56.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:44:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:44:56.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.914684915999999, 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:44:56.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:56.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:44:56.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.809212840000001, 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:44:56.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:44:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:44:56.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.805773265999999, 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.608 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.800987325000001, 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:44:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:44:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:44:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.042829707999999, 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:44:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.042778788, 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:44:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.042713778, 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:44:56.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:44:56.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:56.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:56.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.030759384, 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088343680000000004, 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:44:56.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538058+0000", 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005438109 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.027541169999999, 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:44:56.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.023497104, 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:44:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.638 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/53 ECSubWrite(tid=124, reqid=client.4632.0:725, at_version=71'52, trim_to=0'0, roll_forward_to=68'50))", 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0055166739999999, 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072912799999999998, 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:56.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5000000000000001e-07 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558747+0000", 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042791749999999996 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558803+0000", 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6045000000000003e-05 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558813+0000", 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558899+0000", 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5898999999999999e-05 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558908+0000", 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9570000000000008e-06 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561716+0000", 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028077710000000001 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561758+0000", 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2144000000000002e-05 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0332389260000001, 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079190819999999992, 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526745+0000", 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5100000000000001e-07 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526765+0000", 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0860999999999999e-05 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526775+0000", 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9599999999999995e-06 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526782+0000", 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.426e-06 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528938+0000", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021560450000000001 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528947+0000", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4499999999999993e-06 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534617+0000", 2024-09-15T01:44:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056700489999999999 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534663+0000", 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.604e-05 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:56.649 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0315452709999997, 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079955070000000007, 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528469+0000", 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0893999999999999e-05 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528482+0000", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2745000000000001e-05 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528489+0000", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8920000000000003e-06 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529953+0000", 2024-09-15T01:44:56.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001463696 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529965+0000", 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2004999999999999e-05 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536411+0000", 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064466819999999996 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536433+0000", 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2109000000000001e-05 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 76/53 ECSubWrite(tid=148, reqid=osd.4.0:146, at_version=0'0, trim_to=0'0, roll_forward_to=76'62))", 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0165304239999999, 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0076547359999999997, 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.654 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543464+0000", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0757000000000001e-05 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543472+0000", 2024-09-15T01:44:56.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1810000000000001e-06 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543477+0000", 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3819999999999997e-06 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543557+0000", 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0650999999999997e-05 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543568+0000", 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0998999999999999e-05 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.551078+0000", 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075094150000000002 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.551108+0000", 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9723e-05 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 76/34 ECSubWrite(tid=192, reqid=client.4632.0:766, at_version=76'30, trim_to=0'0, roll_forward_to=55'28))", 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.96780797600000001, 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091339669999999998, 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592177+0000", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7600000000000001e-06 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596007+0000", 2024-09-15T01:44:56.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038296760000000002 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596032+0000", 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978e-05 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596042+0000", 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0582e-05 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:56.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596154+0000", 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111628 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596174+0000", 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0264000000000001e-05 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601267+0000", 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050930120000000001 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601309+0000", 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2067000000000002e-05 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:56.663 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:44:56.665 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:56.663+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:56.766 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:56.759+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:56.776 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:56.776 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:44:56.776 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:44:56.867 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:44:56.860+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:44:56.883 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.883 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:44:56.884 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:44:56.884 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:44:56.884 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:56.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:44:56.968 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:56.963+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:203 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 45.482447845000003, 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010693408999999999, 2024-09-15T01:44:56.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 203 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509398+0000", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509400+0000", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7489999999999999e-06 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509406+0000", 2024-09-15T01:44:56.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029999999999998e-06 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509427+0000", 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1280999999999998e-05 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509435+0000", 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6689999999999995e-06 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509942+0000", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050670200000000004 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510283+0000", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034069100000000001 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520092+0000", 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0098089140000000002 2024-09-15T01:44:57.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 45.482393606000002, 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 45.482374468000003, 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:44:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.473399567999998, 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:44:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:57.016 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.473331905999999, 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:44:57.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.473306164999997, 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:44:57.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.460235355999998, 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:44:57.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:44:57.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.368036240999999, 2024-09-15T01:44:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:44:57.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:44:57.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:537 3.bs0 3:dc585df2:::smithi03327189-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.367970164999999, 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011082247999999999, 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 537 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623876+0000", 2024-09-15T01:44:57.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623877+0000", 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623881+0000", 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623895+0000", 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3091000000000001e-05 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623902+0000", 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.153e-06 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624357+0000", 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045563899999999998 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624496+0000", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013903799999999999 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634734+0000", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01023717 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634849+0000", 2024-09-15T01:44:57.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001154 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.634958+0000", 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109345 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.38872763, 2024-09-15T01:44:57.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:57.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:44:57.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.388594165000001, 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:44:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:44:57.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.467902885, 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:44:57.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:44:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4642714540000004, 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:44:57.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.464197779, 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:44:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:44:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:44:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 72/53 ECSubWrite(tid=159, reqid=osd.0.0:159, at_version=72'62, trim_to=0'0, roll_forward_to=68'60))", 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4639837240000002, 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011041109, 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527862+0000", 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527863+0000", 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.96e-07 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527880+0000", 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7333000000000002e-05 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527889+0000", 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.568e-06 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527894+0000", 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3850000000000001e-06 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528016+0000", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121822 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528025+0000", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0350000000000007e-06 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538848+0000", 2024-09-15T01:44:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010822976 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538904+0000", 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5594000000000003e-05 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 73/66 ECSubWrite(tid=59, reqid=osd.2.0:59, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:44:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.4651075410000001, 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011892049999999999, 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5900000000000002e-07 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526791+0000", 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1916999999999998e-05 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526801+0000", 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7759999999999992e-06 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526805+0000", 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2570000000000001e-06 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527105+0000", 2024-09-15T01:44:57.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029965900000000003 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527122+0000", 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6993000000000001e-05 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538613+0000", 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011490879000000001 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538631+0000", 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8110000000000001e-05 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.4634745450000004, 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016700869, 2024-09-15T01:44:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2e-07 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528400+0000", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7503000000000001e-05 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528410+0000", 2024-09-15T01:44:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0137e-05 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528415+0000", 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4240000000000001e-06 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535553+0000", 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071381459999999997 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535559+0000", 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5210000000000002e-06 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545040+0000", 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094807480000000006 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545073+0000", 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969999999999998e-05 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.083 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 76/53 ECSubWrite(tid=189, reqid=osd.7.0:189, at_version=76'28, trim_to=0'0, roll_forward_to=75'26))", 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.463108257, 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013688799999999999, 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:44:57.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528759+0000", 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1240999999999999e-05 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528776+0000", 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6166000000000001e-05 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528783+0000", 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528831+0000", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7750000000000002e-05 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528838+0000", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0230000000000004e-06 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542366+0000", 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013528014999999999 2024-09-15T01:44:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542427+0000", 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0846e-05 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 76/53 ECSubWrite(tid=145, reqid=osd.4.0:145, at_version=76'94, trim_to=0'0, roll_forward_to=75'92))", 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.4606070870000001, 2024-09-15T01:44:57.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012260155, 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531266+0000", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019e-05 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531281+0000", 2024-09-15T01:44:57.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4214e-05 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531290+0000", 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9290000000000004e-06 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532672+0000", 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013827100000000001 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532682+0000", 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7960000000000007e-06 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543048+0000", 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010366175 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543499+0000", 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045100000000000001 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 76/66 ECSubWrite(tid=67, reqid=osd.2.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=76'21))", 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.4570656900000001, 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013434306999999999, 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:44:57.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534790+0000", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8699999999999998e-06 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534798+0000", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534804+0000", 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7450000000000001e-06 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535012+0000", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020819 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535021+0000", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.473e-06 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548169+0000", 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013148529000000001 2024-09-15T01:44:57.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548215+0000", 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5666e-05 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:44:57.097 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:44:57.099 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:44:57.099 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:57.095+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:57.199 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:57.191+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:57.209 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.209 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:57.209 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:57.209 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:57.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:44:57.300 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:57.295+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:57.319 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.320 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:57.320 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:57.320 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:57.320 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:57.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:44:57.402 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:57.395+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.427 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 44.905203348000001, 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.428 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:44:57.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:44:57.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 44.905145001999998, 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.433 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:44:57.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 44.905066245, 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:44:57.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:57.443 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 34.904886243999997, 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.444 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:44:57.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:44:57.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.894576289, 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:44:57.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:44:57.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.839330542999999, 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:44:57.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.461 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.830488473999999, 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:57.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.467 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.799608852999999, 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.468 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:44:57.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:44:57.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:44:57.474 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.799529485000001, 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:44:57.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.779639104000001, 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.481 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:44:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:44:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.488 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.488 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:57.488 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.778276441999999, 2024-09-15T01:44:57.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:44:57.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:44:57.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.490 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:57.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:44:57.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:44:57.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:44:57.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:44:57.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.498 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:44:57.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:44:57.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.500 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.501 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.501 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.501 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.778261917000002, 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.502 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.504 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:44:57.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:44:57.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:44:57.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:44:57.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:44:57.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:44:57.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:44:57.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:44:57.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:57.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:44:57.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:44:57.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.514 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:547 3.0s0 3:08c156cd:::smithi03327189-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.514 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:57.514 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.778213494999999, 2024-09-15T01:44:57.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0091027969999999993, 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 547 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:57.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645642+0000", 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645643+0000", 2024-09-15T01:44:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.79e-07 2024-09-15T01:44:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645644+0000", 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.699e-06 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645650+0000", 2024-09-15T01:44:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.942e-06 2024-09-15T01:44:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645653+0000", 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.452e-06 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653028+0000", 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073755519999999996 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653094+0000", 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.5718999999999999e-05 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654745+0000", 2024-09-15T01:44:57.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0016511539999999999 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.524 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.755876155999999, 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.525 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.526 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.526 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:44:57.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.526 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:44:57.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:44:57.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:44:57.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.532 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.532 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:44:57.532 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:44:57.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.536 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:44:57.536 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:44:57.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.536 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.537 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:44:57.537 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:44:57.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.537 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:560 3.0s0 3:0899e339:::smithi03327189-38:head [setxattr _header (57) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.752143706999998, 2024-09-15T01:44:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090281700000000003, 2024-09-15T01:44:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 560 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671712+0000", 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671713+0000", 2024-09-15T01:44:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671717+0000", 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1749999999999998e-06 2024-09-15T01:44:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671737+0000", 2024-09-15T01:44:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.948e-05 2024-09-15T01:44:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671744+0000", 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9569999999999996e-06 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676355+0000", 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046115949999999996 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676528+0000", 2024-09-15T01:44:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017328999999999999 2024-09-15T01:44:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677261+0000", 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00073246900000000002 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677982+0000", 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00072084199999999997 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680660+0000", 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002678541 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.680740+0000", 2024-09-15T01:44:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9963000000000007e-05 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.752105833999998, 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:44:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:44:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.906980991999999, 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:44:57.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:44:57.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:675 3.cs0 3:3584cd82:::smithi03327189-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.906863952, 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017344149999999999, 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 675 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516993+0000", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5600000000000001e-06 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517001+0000", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517020+0000", 2024-09-15T01:44:57.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9364999999999998e-05 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517028+0000", 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6229999999999997e-06 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517199+0000", 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017127399999999999 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7309999999999998e-06 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524428+0000", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072231680000000003 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524522+0000", 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4467000000000002e-05 2024-09-15T01:44:57.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534164+0000", 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0096419079999999994 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534243+0000", 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9296999999999997e-05 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534336+0000", 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2608000000000005e-05 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:680 3.0s0 3:0a6b0087:::smithi03327189-4:head [append 0~524288] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.880745659, 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010598392999999999, 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.570 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 680 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:44:57.571 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543113+0000", 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5490000000000001e-06 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544026+0000", 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091366600000000004 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544053+0000", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7015000000000001e-05 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544068+0000", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4977000000000001e-05 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544122+0000", 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3804000000000002e-05 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544197+0000", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5139e-05 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.545372+0000", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001175024 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.547186+0000", 2024-09-15T01:44:57.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001814026 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553517+0000", 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063304659999999999 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553709+0000", 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019172699999999999 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:685 3.0s0 3:0a6b0087:::smithi03327189-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.867099994, 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01007038, 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 685 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.576 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8299999999999997e-07 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556760+0000", 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556766+0000", 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556770+0000", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6109999999999998e-06 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.562875+0000", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061050740000000003 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.563061+0000", 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000186066 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564501+0000", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014400400000000001 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564606+0000", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010522399999999999 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.566826+0000", 2024-09-15T01:44:57.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002219782 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.581 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:57.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:44:57.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:44:57.584 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:44:57.575+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:296: done (0 left) 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:297: read oid 32 snap -1 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:297: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:298: copy_from oid 38 from oid 13 current snap is 23 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 18 current snap is 23 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:299: seq_num 132 ranges {3391488=671744,4063232=573440,4636672=753664,5390336=229376} 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:299: writing smithi03327189-18 from 3391488 to 4063232 tid 1 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:299: writing smithi03327189-18 from 4063232 to 4636672 tid 2 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:299: writing smithi03327189-18 from 4636672 to 5390336 tid 3 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:299: writing smithi03327189-18 from 5390336 to 5619712 tid 4 2024-09-15T01:44:57.585 INFO:tasks.rados.rados.0.smithi033.stdout:300: snap_remove snap 21 2024-09-15T01:44:57.683 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:44:57.676+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:44:57.736 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:44:57.736 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:57.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:44:57.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:44:57.784 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:44:57.775+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:44:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.844 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:44:57.845 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:44:57.845 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:44:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:57.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:44:57.878 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:44:57.885 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:57.880+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:57.897 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:44:57.914 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:44:57.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:44:57.956 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 61.556598285, 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006287684, 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 5 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:44:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394852+0000", 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7460000000000001e-06 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394870+0000", 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8074e-05 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394905+0000", 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4876000000000003e-05 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394923+0000", 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7691000000000001e-05 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395069+0000", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014603000000000001 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395085+0000", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6143000000000001e-05 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398289+0000", 2024-09-15T01:44:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003203759 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398489+0000", 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000200223 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398670+0000", 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000180359 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:57.962 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.400465+0000", 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001795718 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401043+0000", 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057742000000000004 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401080+0000", 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7283000000000002e-05 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401137+0000", 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7361999999999997e-05 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.541700874999997, 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0068396289999999998, 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 7 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.965 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409750+0000", 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0699999999999998e-06 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.966 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409763+0000", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.325e-05 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409805+0000", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1412000000000003e-05 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409839+0000", 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3692999999999997e-05 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.967 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410004+0000", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016576300000000001 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410032+0000", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8076e-05 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413650+0000", 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036180219999999998 2024-09-15T01:44:57.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413889+0000", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00023881400000000001 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.414266+0000", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00037692599999999998 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.415749+0000", 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001483301 2024-09-15T01:44:57.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416329+0000", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057982400000000005 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416478+0000", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014846299999999999 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416587+0000", 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109015 2024-09-15T01:44:57.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:222 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.433188540000003, 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026140312999999998, 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.971 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 222 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.972 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518262+0000", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1090000000000001e-06 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518270+0000", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518286+0000", 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6081000000000001e-05 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518298+0000", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1941e-05 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518644+0000", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034667699999999999 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518650+0000", 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:44:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526775+0000", 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081258210000000001 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526967+0000", 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191991 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.544400+0000", 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017432277999999999 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:223 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.433126627999997, 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027026921999999998, 2024-09-15T01:44:57.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.977 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.977 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.977 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 223 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.978 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518322+0000", 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518328+0000", 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4210000000000003e-06 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518338+0000", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.022e-05 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518343+0000", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.66e-06 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518713+0000", 2024-09-15T01:44:57.980 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036931399999999998 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518715+0000", 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.841e-06 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527539+0000", 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0088243579999999992 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.981 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527752+0000", 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000212888 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017596414000000001 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:224 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:44:57.982 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.433093675999999, 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028715145000000001, 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 224 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.983 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518355+0000", 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:44:57.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518359+0000", 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9550000000000002e-06 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518367+0000", 2024-09-15T01:44:57.985 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518372+0000", 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8420000000000004e-06 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518770+0000", 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039736300000000002 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518772+0000", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2019999999999998e-06 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528208+0000", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094362790000000005 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528403+0000", 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019513399999999999 2024-09-15T01:44:57.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.547069+0000", 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018666242999999999 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.9s4 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.423896925999998, 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0062097970000000004, 2024-09-15T01:44:57.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527552+0000", 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5199999999999996e-07 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528961+0000", 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014095609999999999 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.990 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528973+0000", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1335000000000001e-05 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528990+0000", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7359000000000001e-05 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529050+0000", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0235000000000002e-05 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:57.991 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529062+0000", 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1531e-05 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533731+0000", 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046689110000000004 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533761+0000", 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0113000000000001e-05 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/32 ECSubWriteReply(tid=61, last_complete=33'22, committed=1, applied=1))", 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.423451122000003, 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050195400000000003, 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527998+0000", 2024-09-15T01:44:57.994 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0839999999999999e-06 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528005+0000", 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7209999999999997e-06 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528018+0000", 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3818000000000001e-05 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.995 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528026+0000", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1359999999999999e-06 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533000+0000", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049747359999999996 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533016+0000", 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6045e-05 2024-09-15T01:44:57.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:255 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 39.430792077, 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049454870000000001, 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:57.997 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 255 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:44:57.998 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520660+0000", 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6600000000000001e-06 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520673+0000", 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3406e-05 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:57.999 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520707+0000", 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3902999999999997e-05 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520727+0000", 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0588999999999999e-05 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520839+0000", 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011144199999999999 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.521184+0000", 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034537000000000001 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.525601+0000", 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044171169999999999 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:256 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 39.430673372999998, 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061411549999999997, 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.002 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 256 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520776+0000", 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5e-06 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520786+0000", 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.004 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520807+0000", 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1169999999999999e-05 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520819+0000", 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1589e-05 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522420+0000", 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001600953 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522494+0000", 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4438999999999997e-05 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.526916+0000", 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044215110000000004 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:257 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 39.430613278000003, 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006854008, 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 257 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.007 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520836+0000", 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520844+0000", 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8599999999999993e-06 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520862+0000", 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7859999999999998e-05 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.009 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520872+0000", 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522823+0000", 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019506020000000001 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522947+0000", 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012457899999999999 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.527689+0000", 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004741398 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(osd.1.17:1 3.0s0 3:08c156cd:::smithi03327189-34:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.425519979999997, 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012067648, 2024-09-15T01:44:58.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525932+0000", 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.693e-06 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525943+0000", 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0917000000000001e-05 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.013 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525964+0000", 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0993e-05 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525975+0000", 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1823e-05 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527736+0000", 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001760345 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527967+0000", 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230769 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.537996+0000", 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010029108 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.015 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=50, reqid=client.4632.0:290, at_version=43'17, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.418256012999997, 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061827899999999996, 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.68e-07 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533416+0000", 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000223594 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533428+0000", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649e-05 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.534981+0000", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001553029 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535038+0000", 2024-09-15T01:44:58.018 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6560000000000001e-05 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535050+0000", 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1885999999999999e-05 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539340+0000", 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004289836 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.019 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539375+0000", 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5067999999999997e-05 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=51, reqid=client.4632.0:291, at_version=43'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 35.416414838999998, 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049865860000000003, 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:58.020 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:44:58.021 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9200000000000002e-07 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535160+0000", 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126678 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535171+0000", 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0783e-05 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535178+0000", 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9809999999999997e-06 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535795+0000", 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061714799999999998 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535804+0000", 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0680000000000003e-06 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539989+0000", 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041847079999999997 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.540020+0000", 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0927999999999999e-05 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:393 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 0] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 31.444286311999999, 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077679840000000003, 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 393 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.025 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507165+0000", 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2729999999999998e-06 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507179+0000", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4344e-05 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507215+0000", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5509000000000002e-05 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507230+0000", 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5116000000000001e-05 2024-09-15T01:44:58.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507374+0000", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014417100000000001 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507400+0000", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6222000000000001e-05 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511249+0000", 2024-09-15T01:44:58.028 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038487730000000002 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511443+0000", 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194234 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.514930+0000", 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034863419999999999 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:394 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 31.444193074000001, 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081504530000000002, 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 394 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.030 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507256+0000", 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1230000000000001e-06 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.031 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507260+0000", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3970000000000004e-06 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507278+0000", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8077e-05 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507285+0000", 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507540+0000", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000254541 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507553+0000", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2962999999999999e-05 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511752+0000", 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041991509999999999 2024-09-15T01:44:58.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511891+0000", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013930299999999999 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515075+0000", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0031832549999999999 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515236+0000", 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000161096 2024-09-15T01:44:58.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515405+0000", 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016956100000000001 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:396 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~671744] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.434919384000001, 2024-09-15T01:44:58.035 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014871097, 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 396 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.036 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516531+0000", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8700000000000001e-06 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517417+0000", 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00088526499999999999 2024-09-15T01:44:58.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517445+0000", 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8351e-05 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517458+0000", 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2894000000000001e-05 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.038 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517618+0000", 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000160025 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517871+0000", 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025296800000000001 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.520096+0000", 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022246259999999999 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.039 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523463+0000", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0033674690000000001 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531316+0000", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078531639999999993 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531400+0000", 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3465000000000001e-05 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:397 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~770048] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.427571927999999, 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0082945069999999996, 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 397 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.042 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523879+0000", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2610000000000002e-06 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525106+0000", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266060000000001 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525135+0000", 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525148+0000", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3892e-05 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525342+0000", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019338100000000001 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525610+0000", 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026807099999999999 2024-09-15T01:44:58.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.528197+0000", 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025871959999999999 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531425+0000", 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032278430000000002 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:58.045 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532119+0000", 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069357900000000005 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532171+0000", 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2008000000000001e-05 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:398 3.5s0 3:ac2d3331:::smithi03327189-18:head [setxattr _header (57) in=7b] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:58.046 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.426278546999999, 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073765130000000003, 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 398 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525171+0000", 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.048 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525178+0000", 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1239999999999999e-06 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525193+0000", 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.535e-05 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525203+0000", 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4749999999999999e-06 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.049 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530018+0000", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048148799999999997 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530132+0000", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000114434 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530708+0000", 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057595400000000005 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531935+0000", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266790000000001 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532465+0000", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00052987900000000003 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532546+0000", 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1151999999999998e-05 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:399 3.5s0 3:ac2d3331:::smithi03327189-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.425924077000001, 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090256980000000004, 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 399 2024-09-15T01:44:58.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525525+0000", 2024-09-15T01:44:58.053 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5009999999999999e-06 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525531+0000", 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1120000000000001e-06 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525550+0000", 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8819000000000001e-05 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.054 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525559+0000", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.833e-06 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531733+0000", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061738590000000003 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531869+0000", 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013559600000000001 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532982+0000", 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011133320000000001 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.533176+0000", 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019444500000000001 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.534550+0000", 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013732009999999999 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 48/34 ECSubWriteReply(tid=157, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.425850115999999, 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005543699, 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:44:58.057 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525599+0000", 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:44:58.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525606+0000", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7349999999999999e-06 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525621+0000", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5559000000000001e-05 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525628+0000", 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6560000000000003e-06 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531120+0000", 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0054919829999999998 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531142+0000", 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1671e-05 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:44:58.060 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:44:58.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:44:58.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:44:58.061 INFO:tasks.thrashosds.thrasher:inject_pause on osd.8 2024-09-15T01:44:58.061 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T01:44:58.061 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T01:44:58.061 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok config set filestore_inject_stall 3 2024-09-15T01:44:58.064 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:44:58.064 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:44:58.064 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:44:58.064 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:44:58.064 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:44:58.064 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:58.056+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:58.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:44:58.071 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:44:58.094 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:44:58.115 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:44:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:44:58.164 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:58.155+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:58.171 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-09-15T01:44:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "success": "" 2024-09-15T01:44:58.171 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-09-15T01:44:58.173 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.173 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:44:58.265 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:58.259+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:58.286 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.287 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:44:58.365 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:58.363+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:58.401 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.401 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:44:58.467 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:58.464+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:58.507 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.507 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:44:58.548 INFO:tasks.rados.rados.0.smithi033.stdout:300: done (3 left) 2024-09-15T01:44:58.549 INFO:tasks.rados.rados.0.smithi033.stdout:301: snap_create 2024-09-15T01:44:58.549 INFO:tasks.rados.rados.0.smithi033.stdout:297: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:44:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:44:58.568 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:44:58.559+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:44:58.623 INFO:tasks.rados.rados.0.smithi033.stdout:298: finishing copy_from racing read to smithi03327189-38 2024-09-15T01:44:58.623 INFO:tasks.rados.rados.0.smithi033.stdout:298: finishing copy_from to smithi03327189-38 2024-09-15T01:44:58.623 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 30 (ObjNum 99 snap 13 seq_num 99) dirty exists 2024-09-15T01:44:58.623 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 1 to smithi03327189-18 2024-09-15T01:44:58.623 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 2 to smithi03327189-18 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 3 to smithi03327189-18 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 4 to smithi03327189-18 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 5 to smithi03327189-18 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:299: finishing write tid 6 to smithi03327189-18 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 135 (ObjNum 132 snap 23 seq_num 132) dirty exists 2024-09-15T01:44:58.624 INFO:tasks.rados.rados.0.smithi033.stdout:299: left oid 18 (ObjNum 132 snap 23 seq_num 132) 2024-09-15T01:44:58.626 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.626 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:44:58.669 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:58.663+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:58.734 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.734 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.743 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:44:58.771 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:58.764+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:58.848 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.849 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.859 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:44:58.872 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:58.863+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:58.960 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:58.961 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:58.973 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:44:58.974 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:44:58.968+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:44:59.075 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.076 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.076 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:59.068+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:59.084 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:44:59.176 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:59.172+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:59.184 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.184 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.193 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:44:59.277 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:44:59.275+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:44:59.292 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.293 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.302 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:44:59.378 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:59.371+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:59.400 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.400 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.409 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:44:59.478 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:44:59.471+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:297: done (3 left) 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:298: done (2 left) 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:299: done (1 left) 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:301: done (0 left) 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 45 current snap is 24 2024-09-15T01:44:59.555 INFO:tasks.rados.rados.0.smithi033.stdout:302: seq_num 133 ranges {0=688128,688128=720896,1409024=475136,1884160=360448} 2024-09-15T01:44:59.556 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.557 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.563 INFO:tasks.rados.rados.0.smithi033.stdout:302: writing smithi03327189-45 from 0 to 688128 tid 1 2024-09-15T01:44:59.566 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:44:59.567 INFO:tasks.rados.rados.0.smithi033.stdout:302: writing smithi03327189-45 from 688128 to 1409024 tid 2 2024-09-15T01:44:59.570 INFO:tasks.rados.rados.0.smithi033.stdout:302: writing smithi03327189-45 from 1409024 to 1884160 tid 3 2024-09-15T01:44:59.573 INFO:tasks.rados.rados.0.smithi033.stdout:302: writing smithi03327189-45 from 1884160 to 2244608 tid 4 2024-09-15T01:44:59.573 INFO:tasks.rados.rados.0.smithi033.stdout:303: rmattr oid 15 current snap is 24 2024-09-15T01:44:59.573 INFO:tasks.rados.rados.0.smithi033.stdout:303: done (1 left) 2024-09-15T01:44:59.573 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 5 current snap is 24 2024-09-15T01:44:59.573 INFO:tasks.rados.rados.0.smithi033.stdout:304: seq_num 134 ranges {2129920=704512,2834432=475136,3309568=573440,3883008=507904} 2024-09-15T01:44:59.579 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:59.572+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:44:59.589 INFO:tasks.rados.rados.0.smithi033.stdout:304: writing smithi03327189-5 from 2129920 to 2834432 tid 1 2024-09-15T01:44:59.592 INFO:tasks.rados.rados.0.smithi033.stdout:304: writing smithi03327189-5 from 2834432 to 3309568 tid 2 2024-09-15T01:44:59.596 INFO:tasks.rados.rados.0.smithi033.stdout:304: writing smithi03327189-5 from 3309568 to 3883008 tid 3 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:304: writing smithi03327189-5 from 3883008 to 4390912 tid 4 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:305: copy_from oid 34 from oid 43 current snap is 24 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 1 to smithi03327189-45 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 2 to smithi03327189-45 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 3 to smithi03327189-45 2024-09-15T01:44:59.599 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 4 to smithi03327189-45 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 5 to smithi03327189-45 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:302: finishing write tid 6 to smithi03327189-45 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 45 v 142 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:302: left oid 45 (ObjNum 133 snap 24 seq_num 133) 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 1 to smithi03327189-5 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:302: done (2 left) 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:306: rollback oid 49 current snap is 24 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 49 to 23 2024-09-15T01:44:59.600 INFO:tasks.rados.rados.0.smithi033.stdout:307: snap_create 2024-09-15T01:44:59.601 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 2 to smithi03327189-5 2024-09-15T01:44:59.606 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 3 to smithi03327189-5 2024-09-15T01:44:59.607 INFO:tasks.rados.rados.0.smithi033.stdout:306: finishing rollback tid 0 to smithi03327189-49 2024-09-15T01:44:59.607 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 4 to smithi03327189-5 2024-09-15T01:44:59.607 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 5 to smithi03327189-5 2024-09-15T01:44:59.607 INFO:tasks.rados.rados.0.smithi033.stdout:306: finishing rollback tid 1 to smithi03327189-49 2024-09-15T01:44:59.608 INFO:tasks.rados.rados.0.smithi033.stdout:306: finishing rollback tid 2 to smithi03327189-49 2024-09-15T01:44:59.608 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 49 v 49 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:44:59.609 INFO:tasks.rados.rados.0.smithi033.stdout:304: finishing write tid 6 to smithi03327189-5 2024-09-15T01:44:59.609 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 5 v 148 (ObjNum 134 snap 24 seq_num 134) dirty exists 2024-09-15T01:44:59.609 INFO:tasks.rados.rados.0.smithi033.stdout:304: left oid 5 (ObjNum 134 snap 24 seq_num 134) 2024-09-15T01:44:59.611 INFO:tasks.rados.rados.0.smithi033.stdout:305: finishing copy_from racing read to smithi03327189-34 2024-09-15T01:44:59.613 INFO:tasks.rados.rados.0.smithi033.stdout:305: finishing copy_from to smithi03327189-34 2024-09-15T01:44:59.613 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 34 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:44:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.671 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:44:59.680 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:44:59.671+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:44:59.762 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.762 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.771 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:44:59.816 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:44:59.811+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:44:59.881 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:44:59.875+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:44:59.917 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:44:59.917 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:44:59.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:44:59.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:44:59.976+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:00.023 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:00.023 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:00.032 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:45:00.082 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:00.075+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:00.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:45:00.131 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:00.131 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:00.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:45:00.183 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:00.179+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:00.285 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:00.280+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:00.285 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:00.289 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:00.289 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:00.289 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:00.289 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:00.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:45:00.306 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:00.327 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:00.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:00.368 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:00.386 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:00.379+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:00.388 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:00.388 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:00.388 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:00.388 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:00.389 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:00.391 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:00.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:45:00.415 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:00.438 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:00.460 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:00.481 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:00.486 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:00.479+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:00.504 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.987140689999997, 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:45:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:45:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.984709291999998, 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:45:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.984470477000002, 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:45:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:45:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.984439094000003, 2024-09-15T01:45:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:45:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.005231664, 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:45:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.997317262000003, 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:45:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:45:00.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.555 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.997287999000001, 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:45:00.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:45:00.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:45:00.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.997712679999999, 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:45:00.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:45:00.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.995474219000002, 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:45:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:45:00.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:45:00.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.994904550000001, 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:45:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.994845581999996, 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:45:00.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.006947373999999, 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:45:00.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:00.594 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.006853595999999, 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:00.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:45:00.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:00.600 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.940209482, 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:45:00.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=117, reqid=osd.4.0:117, at_version=68'74, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.999118205, 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011721416, 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:00.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514471+0000", 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:00.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514541+0000", 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0955000000000001e-05 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514567+0000", 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.597e-05 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514584+0000", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6463000000000001e-05 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514727+0000", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014272899999999999 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514754+0000", 2024-09-15T01:45:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7290000000000001e-05 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526159+0000", 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011404621 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526191+0000", 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2840999999999999e-05 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/53 ECSubWrite(tid=118, reqid=osd.4.0:118, at_version=68'50, trim_to=0'0, roll_forward_to=67'48))", 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.998710956, 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012184878999999999, 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514879+0000", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514938+0000", 2024-09-15T01:45:00.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9142999999999999e-05 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514963+0000", 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5165999999999999e-05 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515032+0000", 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8602999999999994e-05 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515160+0000", 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012877200000000001 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515188+0000", 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7313000000000001e-05 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527007+0000", 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011819079999999999 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527062+0000", 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5328999999999998e-05 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=68'76, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.996656572999999, 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014580965, 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516933+0000", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6249999999999999e-06 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516977+0000", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3551999999999999e-05 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517002+0000", 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4762000000000001e-05 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517016+0000", 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.401e-05 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517789+0000", 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077353199999999995 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517813+0000", 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3748e-05 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531475+0000", 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013662636000000001 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531513+0000", 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7100000000000001e-05 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:753 3.1s0 3:86165cc9:::smithi03327189-11:head [copy-from ver 13] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 7.9122365930000003, 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022114891000000001, 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.621 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 753 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:00.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601354+0000", 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6000000000000001e-06 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601366+0000", 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601396+0000", 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9901999999999999e-05 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601406+0000", 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601519+0000", 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112546 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601677+0000", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015800100000000001 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.614051+0000", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012374232000000001 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617994+0000", 2024-09-15T01:45:00.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039432089999999996 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623415+0000", 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054204240000000001 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623466+0000", 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1972000000000003e-05 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:754 3.1s0 3:86165cc9:::smithi03327189-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 7.9121569459999996, 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015926781000000001, 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 754 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601433+0000", 2024-09-15T01:45:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4270000000000001e-06 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601437+0000", 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2899999999999996e-06 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601452+0000", 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4939999999999999e-05 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601461+0000", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601979+0000", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051790899999999999 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.602057+0000", 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8237000000000003e-05 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617204+0000", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015147361999999999 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617280+0000", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6318999999999999e-05 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617358+0000", 2024-09-15T01:45:00.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7570000000000004e-05 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:6 3.6s0 3:6579c50e:::smithi03327189-13:head [assert-version v30,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.9938307800000001, 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011523049, 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519761+0000", 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519774+0000", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3550000000000001e-05 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519826+0000", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519847+0000", 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0890000000000002e-05 2024-09-15T01:45:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.520007+0000", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000159801 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.520028+0000", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1696999999999999e-05 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.525500+0000", 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054716089999999997 2024-09-15T01:45:00.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.525646+0000", 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014582700000000001 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.531280+0000", 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056347089999999999 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:00.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:00.637 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:00.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:45:00.638 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:00.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:00.641 INFO:tasks.rados.rados.0.smithi033.stdout:304: done (3 left) 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:305: done (2 left) 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:306: done (1 left) 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:307: done (0 left) 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 18 current snap is 25 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:308: seq_num 135 ranges {5619712=720896,6340608=622592,6963200=688128,7651328=245760} 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:308: writing smithi03327189-18 from 5619712 to 6340608 tid 1 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:308: writing smithi03327189-18 from 6340608 to 6963200 tid 2 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:308: writing smithi03327189-18 from 6963200 to 7651328 tid 3 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:308: writing smithi03327189-18 from 7651328 to 7897088 tid 4 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:309: rollback oid 33 current snap is 25 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 33 to 23 2024-09-15T01:45:00.642 INFO:tasks.rados.rados.0.smithi033.stdout:310: snap_remove snap 24 2024-09-15T01:45:00.643 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:00.631+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:00.739 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:00.735+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:00.785 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:00.785 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:00.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:45:00.840 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:00.836+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:00.892 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:00.893 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:00.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:45:00.944 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:00.940+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:01.041 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:01.035+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.524916748000003, 2024-09-15T01:45:01.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:45:01.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.524885804, 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:45:01.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:45:01.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:45:01.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.528005983, 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:45:01.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:45:01.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:45:01.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.527936451000002, 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:45:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:45:01.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.527913914999999, 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:45:01.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:45:01.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.527846723000003, 2024-09-15T01:45:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:45:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:45:01.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.527777198000003, 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:45:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.092 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.527535120000003, 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:01.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.527481686000002, 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:45:01.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:45:01.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.527409507000002, 2024-09-15T01:45:01.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:45:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:45:01.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:511 3.4s0 3:20577364:::smithi03327189-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:45:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.409929475999999, 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014490421999999999, 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 511 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:45:01.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636012+0000", 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636013+0000", 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3799999999999997e-07 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636015+0000", 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2869999999999999e-06 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636020+0000", 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.34e-06 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.636022+0000", 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3779999999999999e-06 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640680+0000", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046585400000000001 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.640820+0000", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014002599999999999 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647530+0000", 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067096409999999997 2024-09-15T01:45:01.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647618+0000", 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7530999999999999e-05 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.650503+0000", 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028851409999999999 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.521558828, 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:45:01.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:45:01.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.516642006000001, 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.120 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:45:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:45:01.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 73/53 ECSubWrite(tid=132, reqid=osd.4.0:132, at_version=73'84, trim_to=0'0, roll_forward_to=72'82))", 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5290560370000001, 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015282341, 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:45:01.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516946+0000", 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9145999999999999e-05 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516976+0000", 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0236e-05 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516993+0000", 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517168+0000", 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017460800000000001 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517197+0000", 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8813e-05 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532115+0000", 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014917629 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532168+0000", 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3596e-05 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:740 3.4s0 3:20577364:::smithi03327189-24:head [copy-from ver 0] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:01.130 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5263727350000007, 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016305848000000001, 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 740 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519573+0000", 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-06 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519590+0000", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519639+0000", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9432999999999999e-05 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519658+0000", 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9083e-05 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521437+0000", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017789539999999999 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521452+0000", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4952e-05 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530454+0000", 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.009001963 2024-09-15T01:45:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530578+0000", 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012395599999999999 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535875+0000", 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052969169999999999 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:741 3.4s0 3:20577364:::smithi03327189-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5261954469999992, 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016411578999999999, 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 741 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519748+0000", 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.04e-06 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519758+0000", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5929999999999995e-06 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519795+0000", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6860000000000003e-05 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519810+0000", 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5082999999999999e-05 2024-09-15T01:45:01.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521558+0000", 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017476 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521568+0000", 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0344e-05 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530788+0000", 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092198539999999995 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530875+0000", 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7698e-05 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535967+0000", 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918409999999997 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536066+0000", 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8281000000000003e-05 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536158+0000", 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2385000000000004e-05 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.141 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=183, reqid=client.4632.0:738, at_version=73'21, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5224020039999999, 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018357772000000001, 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523541+0000", 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.24e-06 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523599+0000", 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7500999999999997e-05 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523628+0000", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9193999999999999e-05 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523640+0000", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2276000000000001e-05 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531608+0000", 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079682079999999992 2024-09-15T01:45:01.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531619+0000", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0293e-05 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541866+0000", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010247114999999999 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541898+0000", 2024-09-15T01:45:01.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1945e-05 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 73/34 ECSubWrite(tid=58, reqid=osd.2.0:58, at_version=73'43, trim_to=0'0, roll_forward_to=72'41))", 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5196626809999998, 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015869550999999999, 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526280+0000", 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999996e-07 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526306+0000", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6282e-05 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526322+0000", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5977999999999998e-05 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526330+0000", 2024-09-15T01:45:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.191e-06 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526451+0000", 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120526 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526459+0000", 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542088+0000", 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015629295000000001 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542149+0000", 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0247000000000001e-05 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=184, reqid=client.4632.0:739, at_version=73'22, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5185300989999995, 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014513999, 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527412+0000", 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527413+0000", 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9100000000000002e-07 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527437+0000", 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3961e-05 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527451+0000", 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3966e-05 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527464+0000", 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3030999999999999e-05 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.533825+0000", 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063618429999999998 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.533837+0000", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1871999999999999e-05 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541902+0000", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.008064903 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541926+0000", 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3532000000000001e-05 2024-09-15T01:45:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5184692379999998, 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014773232000000001, 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527518+0000", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4802000000000002e-05 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527528+0000", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8640000000000008e-06 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527534+0000", 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7590000000000003e-06 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534080+0000", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065466159999999999 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534092+0000", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1953999999999999e-05 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542223+0000", 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081309510000000008 2024-09-15T01:45:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542246+0000", 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2776999999999999e-05 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.159 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:45:01.161 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:01.159+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:01.261 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:01.259+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:01.265 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.265 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:01.265 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:01.265 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:45:01.362 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:01.359+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:01.370 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.370 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:01.370 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:01.370 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:01.370 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:45:01.463 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:01.459+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:01.487 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.487 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:01.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/34 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=38'32, trim_to=0'0, roll_forward_to=37'30))", 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.961652436999998, 2024-09-15T01:45:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077563290000000002, 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519257+0000", 2024-09-15T01:45:01.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519258+0000", 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9999999999999998e-07 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519318+0000", 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0449999999999999e-05 2024-09-15T01:45:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519358+0000", 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9841e-05 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.519377+0000", 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9063e-05 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520207+0000", 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000829662 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.520235+0000", 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8767000000000001e-05 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.526988+0000", 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067525529999999997 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:17.527013+0000", 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5493000000000001e-05 2024-09-15T01:45:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.959166912999997, 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:45:01.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:45:01.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.951663661000001, 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:45:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:516 3.es0 3:7de15fb4:::smithi03327189-49:head [append 0~638976] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.835531899999999, 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084871579999999999, 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 516 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645378+0000", 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645379+0000", 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6959999999999999e-06 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645705+0000", 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032527199999999999 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645725+0000", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0452e-05 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.645732+0000", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3540000000000002e-06 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646455+0000", 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072240300000000004 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.646578+0000", 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000122879 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.647923+0000", 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013450739999999999 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653256+0000", 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053331070000000001 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653824+0000", 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056813399999999998 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653865+0000", 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0787000000000001e-05 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.513 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.831103290000001, 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:45:01.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.515 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:01.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.835611240999999, 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.730139165000001, 2024-09-15T01:45:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:45:01.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:01.533 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.726699590999999, 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:01.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:45:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.721913650000001, 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:45:01.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:01.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.963756032999999, 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.963705113, 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:45:01.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:45:01.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.963640102999999, 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:45:01.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.951685708999999, 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088343680000000004, 2024-09-15T01:45:01.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:45:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538058+0000", 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005438109 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.948467494999999, 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:01.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.944423429, 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:45:01.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:45:01.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 71/53 ECSubWrite(tid=124, reqid=client.4632.0:725, at_version=71'52, trim_to=0'0, roll_forward_to=68'50))", 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.926442999000001, 2024-09-15T01:45:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072912799999999998, 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.554467+0000", 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5000000000000001e-07 2024-09-15T01:45:01.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558747+0000", 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042791749999999996 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558803+0000", 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6045000000000003e-05 2024-09-15T01:45:01.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558813+0000", 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558899+0000", 2024-09-15T01:45:01.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5898999999999999e-05 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.558908+0000", 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9570000000000008e-06 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561716+0000", 2024-09-15T01:45:01.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028077710000000001 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:50.561758+0000", 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2144000000000002e-05 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9541652509999992, 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079190819999999992, 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526744+0000", 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526745+0000", 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5100000000000001e-07 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526765+0000", 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0860999999999999e-05 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526775+0000", 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9599999999999995e-06 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.526782+0000", 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.426e-06 2024-09-15T01:45:01.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528938+0000", 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021560450000000001 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528947+0000", 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4499999999999993e-06 2024-09-15T01:45:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534617+0000", 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056700489999999999 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.534663+0000", 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.604e-05 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9524715960000005, 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079955070000000007, 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:45:01.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528438+0000", 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8400000000000005e-07 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528469+0000", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0893999999999999e-05 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528482+0000", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2745000000000001e-05 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.528489+0000", 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8920000000000003e-06 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529953+0000", 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001463696 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.529965+0000", 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2004999999999999e-05 2024-09-15T01:45:01.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536411+0000", 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064466819999999996 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.536433+0000", 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2109000000000001e-05 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs5 76/53 ECSubWrite(tid=148, reqid=osd.4.0:146, at_version=0'0, trim_to=0'0, roll_forward_to=76'62))", 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.9374567489999999, 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0076547359999999997, 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543453+0000", 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543464+0000", 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0757000000000001e-05 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543472+0000", 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1810000000000001e-06 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543477+0000", 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3819999999999997e-06 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543557+0000", 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0650999999999997e-05 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543568+0000", 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0998999999999999e-05 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.551078+0000", 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075094150000000002 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.551108+0000", 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9723e-05 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.592 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 76/34 ECSubWrite(tid=192, reqid=client.4632.0:766, at_version=76'30, trim_to=0'0, roll_forward_to=55'28))", 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.8887343009999995, 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091339669999999998, 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:01.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592177+0000", 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7600000000000001e-06 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596007+0000", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038296760000000002 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596032+0000", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978e-05 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596042+0000", 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0582e-05 2024-09-15T01:45:01.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596154+0000", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111628 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596174+0000", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0264000000000001e-05 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601267+0000", 2024-09-15T01:45:01.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050930120000000001 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601309+0000", 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2067000000000002e-05 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.597 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:45:01.600 INFO:tasks.rados.rados.0.smithi033.stdout:310: done (2 left) 2024-09-15T01:45:01.600 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 17 current snap is 25 2024-09-15T01:45:01.600 INFO:tasks.rados.rados.0.smithi033.stdout:309: finishing rollback tid 0 to smithi03327189-33 2024-09-15T01:45:01.600 INFO:tasks.rados.rados.0.smithi033.stdout:311: seq_num 136 ranges {0=737280,737280=770048,1507328=540672,2048000=245760} 2024-09-15T01:45:01.600 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:45:01.601 INFO:tasks.rados.rados.0.smithi033.stdout:311: writing smithi03327189-17 from 0 to 737280 tid 1 2024-09-15T01:45:01.601 INFO:tasks.rados.rados.0.smithi033.stdout:311: writing smithi03327189-17 from 737280 to 1507328 tid 2 2024-09-15T01:45:01.601 INFO:tasks.rados.rados.0.smithi033.stdout:311: writing smithi03327189-17 from 1507328 to 2048000 tid 3 2024-09-15T01:45:01.601 INFO:tasks.rados.rados.0.smithi033.stdout:311: writing smithi03327189-17 from 2048000 to 2293760 tid 4 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 1 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:309: finishing rollback tid 1 to smithi03327189-33 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:309: finishing rollback tid 2 to smithi03327189-33 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 33 v 58 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 2 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 3 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 4 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 5 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: finishing write tid 6 to smithi03327189-18 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 154 (ObjNum 135 snap 25 seq_num 135) dirty exists 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: left oid 18 (ObjNum 135 snap 25 seq_num 135) 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:308: done (2 left) 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:309: done (1 left) 2024-09-15T01:45:01.602 INFO:tasks.rados.rados.0.smithi033.stdout:312: read oid 3 snap -1 2024-09-15T01:45:01.603 INFO:tasks.rados.rados.0.smithi033.stdout:312: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:45:01.603 INFO:tasks.rados.rados.0.smithi033.stdout:313: snap_remove snap 23 2024-09-15T01:45:01.603 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:01.592+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:01.700 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:01.696+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:01.754 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.754 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:01.754 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:01.754 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:45:01.808 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:01.804+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:01.861 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.861 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:01.861 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:01.861 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:01.861 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:01.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:45:01.901 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:01.899+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:204 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 50.459220461999998, 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011163206, 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 204 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509453+0000", 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6899999999999997e-07 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509456+0000", 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.32e-06 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509463+0000", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509467+0000", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.189e-06 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510757+0000", 2024-09-15T01:45:01.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012901010000000001 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.510851+0000", 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3477e-05 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.520616+0000", 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097650059999999997 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 50.459201323999999, 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.979 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:45:01.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.450226424, 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:45:01.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:45:01.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:45:01.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.450158762000001, 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:45:01.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:45:01.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.450133020999999, 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:01.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:45:01.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.1s3 43/34 ECSubWrite(tid=52, reqid=client.4632.0:292, at_version=43'19, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.437062212000001, 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011113235000000001, 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:01.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531611+0000", 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531621+0000", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0403e-05 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531658+0000", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7119999999999997e-05 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531676+0000", 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7501999999999999e-05 2024-09-15T01:45:02.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531683+0000", 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1760000000000004e-06 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531902+0000", 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021900899999999999 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531912+0000", 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5899999999999997e-06 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542684+0000", 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010772161000000001 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542724+0000", 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0274000000000001e-05 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.344863097000001, 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:45:02.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:45:02.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.009 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.365554486000001, 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:45:02.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.365421021, 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:45:02.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:02.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:45:02.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.444729741, 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:45:02.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:02.026 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.441098309999999, 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:02.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:45:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.441024635, 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:45:02.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 73/66 ECSubWrite(tid=59, reqid=osd.2.0:59, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-15T01:45:02.035 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4419343970000007, 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011892049999999999, 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5900000000000002e-07 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526791+0000", 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1916999999999998e-05 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526801+0000", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7759999999999992e-06 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526805+0000", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2570000000000001e-06 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527105+0000", 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029965900000000003 2024-09-15T01:45:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527122+0000", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6993000000000001e-05 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538613+0000", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011490879000000001 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538631+0000", 2024-09-15T01:45:02.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8110000000000001e-05 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4403014009999993, 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016700869, 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:02.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2e-07 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528400+0000", 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7503000000000001e-05 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528410+0000", 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0137e-05 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528415+0000", 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4240000000000001e-06 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535553+0000", 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071381459999999997 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535559+0000", 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5210000000000002e-06 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545040+0000", 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094807480000000006 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545073+0000", 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969999999999998e-05 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.044 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 76/53 ECSubWrite(tid=189, reqid=osd.7.0:189, at_version=76'28, trim_to=0'0, roll_forward_to=75'26))", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.4399351129999998, 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013688799999999999, 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528759+0000", 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1240999999999999e-05 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528776+0000", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6166000000000001e-05 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528783+0000", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528831+0000", 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7750000000000002e-05 2024-09-15T01:45:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528838+0000", 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0230000000000004e-06 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542366+0000", 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013528014999999999 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542427+0000", 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0846e-05 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 76/53 ECSubWrite(tid=145, reqid=osd.4.0:145, at_version=76'94, trim_to=0'0, roll_forward_to=75'92))", 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.4374339430000003, 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012260155, 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.049 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531266+0000", 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019e-05 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531281+0000", 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4214e-05 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531290+0000", 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9290000000000004e-06 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532672+0000", 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013827100000000001 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532682+0000", 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7960000000000007e-06 2024-09-15T01:45:02.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543048+0000", 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010366175 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543499+0000", 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045100000000000001 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 76/66 ECSubWrite(tid=67, reqid=osd.2.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=76'21))", 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.433892546, 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013434306999999999, 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534790+0000", 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8699999999999998e-06 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534798+0000", 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534804+0000", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7450000000000001e-06 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535012+0000", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020819 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535021+0000", 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.473e-06 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548169+0000", 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013148529000000001 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548215+0000", 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5666e-05 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.059 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:807 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.365128747, 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012342736999999999, 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 807 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603547+0000", 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9380000000000001e-06 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603562+0000", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4431e-05 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603604+0000", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603622+0000", 2024-09-15T01:45:02.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7706e-05 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603719+0000", 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7546000000000004e-05 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603728+0000", 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608524+0000", 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047951790000000001 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608636+0000", 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011230600000000001 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.609358+0000", 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072179799999999995 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613504+0000", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041461270000000003 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615819+0000", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023155229999999999 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615887+0000", 2024-09-15T01:45:02.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7708999999999998e-05 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:808 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [rollback 30] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3649979370000001, 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012866819999999999, 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 808 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603677+0000", 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603684+0000", 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3220000000000004e-06 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603705+0000", 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1056999999999998e-05 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603716+0000", 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1073999999999999e-05 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603799+0000", 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3174e-05 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603803+0000", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.422e-06 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610165+0000", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006361841 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610363+0000", 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019824 2024-09-15T01:45:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.611276+0000", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091360300000000003 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613506+0000", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022295459999999998 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616474+0000", 2024-09-15T01:45:02.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029682599999999999 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616542+0000", 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7694999999999995e-05 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:809 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:02.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.364887967, 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015369404999999999, 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 809 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603787+0000", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603793+0000", 2024-09-15T01:45:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.951e-06 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603822+0000", 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8274000000000001e-05 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603833+0000", 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1243000000000001e-05 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603863+0000", 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0425000000000001e-05 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603867+0000", 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2030000000000001e-06 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612025+0000", 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081579340000000004 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612192+0000", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016787099999999999 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613063+0000", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087049600000000001 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615910+0000", 2024-09-15T01:45:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028468719999999999 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619073+0000", 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031635119999999998 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619154+0000", 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1206999999999997e-05 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:02.079 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:02.079 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:02.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:45:02.081 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:02.075+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:02.181 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:02.175+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:02.192 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:02.192 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:02.192 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:02.192 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:02.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:45:02.282 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:02.275+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:02.305 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:02.305 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:02.305 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:02.305 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:02.305 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:02.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:45:02.382 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:02.375+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:02.417 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:02.417 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:45:02.417 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:45:02.417 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:45:02.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 49.893546422, 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.418 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:45:02.420 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.421 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.422 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 49.893488075999997, 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.423 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.424 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:45:02.426 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 49.893409319, 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:45:02.432 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:302 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 39.893229318000003, 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013110058000000001, 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 302 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518970+0000", 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.435 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518972+0000", 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.65e-06 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.518984+0000", 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1440000000000001e-05 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519022+0000", 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919000000000001e-05 2024-09-15T01:45:02.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519036+0000", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4387e-05 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519161+0000", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012514700000000001 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.519457+0000", 2024-09-15T01:45:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00029583099999999998 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.521070+0000", 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001613312 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.523270+0000", 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002199711 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.438 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.531865+0000", 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085952690000000009 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.532080+0000", 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00021439199999999999 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:480 3.0s0 3:08ecf469:::smithi03327189-6:head [create,setxattr \\\\jePWq]ZOWNpZOUZYVPc^lpj`fUUtsQVY\\NhUQ] (1719) in=40b,setxattr aoaWl[dPlh\\XpV^jtfVhPOYqn\\NhXU[najjbRNWd (10033) in=40b,setxattr `V_nomOPOmgVgNrNl`jNVs_kSmRdSl^ioTPc\\^eY (19716) in=40b,setxattr lecebNUs]cdSasSblV`sWfcidcPc^bU (279) in=31b,setxattr cS]cSWa_lh^jVbWphdgpXmgYYcnank`UpmobcZUa (4768) in=40b,setxattr hmNTeiNaYhNQOPaRafS\\N[Q\\lni (279) in=27b,setxattr sVZOOgbVN`ejVfhSRZcaTtqnfdgPbpaYeNQbSf`c (9120) in=40b,setxattr dobiNs^RXTPjhaQtOaipXXfdWQYafcajclaW`TiO (1050) in=40b,setxattr nqoachfoWSY`OSgs_VoZTbhe]egYaQiZl[RTphVr (1821) in=40b,setxattr g\\scUhe]\\_jRp^SnXga_\\VtssTTbbaj^cSq\\ffmW (595) in=40b,setxattr UXV`NXQYomt]eTei[ksqfOr_cm (279) in=26b,setxattr Y_]abT\\^UYakbfWl]\\dTibZaSYj^aa^oUQtWpaQa (771) in=40b,setxattr ]Xga]WS^XrRqqrQSfTaoOgSYam_`mPmNXg`tNYmT (1789) in=40b,setxattr WZUeZrcUPqodUr`\\UmkgYpXpr[W^ceZRfhO\\nRaj (17824) in=40b,setxattr \\pXoTRPmQWkcio`^a^X`gNTan_Rcop[gm_ecWliQ (946) in=40b,setxattr NnZhqU`cnT_edhVicNgm`a[XR\\ZOoNaZshtl[Paq (15156) in=40b,setxattr lhackQeZc^aYSZgjq][O[rfV\\cgsZ[tah_Q_cX\\h (8420) in=40b,setxattr PWN]cjYVneeRNnlYgcijosg]e^_oTP_[YlT[_gX^ (1649) in=40b,setxattr WtRNQ`NYWWmoWpjVqhb]\\apOq]RTUbOfshmUmtTR (8628) in=40b,setxattr hmX_UbmcY[ROefRbcXXg\\okbS`bTONmsnTN[ceSt (19508) in=40b] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:02.439 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.882919362999999, 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010574961000000001, 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 480 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.440 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529280+0000", 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529283+0000", 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1879999999999998e-06 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.441 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529484+0000", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020126200000000001 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529518+0000", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3575999999999998e-05 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529536+0000", 2024-09-15T01:45:02.442 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8691000000000001e-05 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.529642+0000", 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000105429 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.530140+0000", 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00049821100000000003 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.443 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.533202+0000", 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030621509999999999 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.536938+0000", 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0037361450000000002 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539753+0000", 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00281518 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:36.539854+0000", 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000101128 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:521 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [append 0~573440] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.827673616999999, 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012696063, 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 521 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.446 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584525+0000", 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.584527+0000", 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9300000000000002e-06 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585365+0000", 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00083761800000000004 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585380+0000", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4916000000000001e-05 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585392+0000", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1914e-05 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585488+0000", 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5877000000000004e-05 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.585670+0000", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018283399999999999 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.587033+0000", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013629079999999999 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.589166+0000", 2024-09-15T01:45:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0021326209999999999 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597095+0000", 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0079288870000000008 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.597221+0000", 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126558 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:525 3.0s0 3:0dfa6ad8:::smithi03327189-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.818831547999999, 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.009667531, 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 525 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.451 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593367+0000", 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.452 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593368+0000", 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1600000000000001e-07 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593371+0000", 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2270000000000002e-06 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593380+0000", 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7909999999999993e-06 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.453 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.593390+0000", 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.781e-06 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598191+0000", 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048009649999999999 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.454 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.598276+0000", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5054000000000003e-05 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601506+0000", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032298510000000002 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.601623+0000", 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011707 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.603035+0000", 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001412076 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:531 3.0s0 3:0a6b0087:::smithi03327189-4:head [copy-from ver 44] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.787951927000002, 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019996634999999999, 2024-09-15T01:45:02.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 531 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:45:02.457 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624247+0000", 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624249+0000", 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.08e-06 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.458 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624259+0000", 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624289+0000", 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0366999999999999e-05 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624302+0000", 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2840000000000001e-05 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624418+0000", 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000115752 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624601+0000", 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000183609 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.635935+0000", 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011333302999999999 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.640157+0000", 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042220440000000003 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644072+0000", 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0039152919999999999 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644244+0000", 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000171464 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:532 3.0s0 3:0a6b0087:::smithi03327189-4:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.787872559, 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012773029, 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.462 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 532 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624326+0000", 2024-09-15T01:45:02.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624328+0000", 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.24e-06 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624335+0000", 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2459999999999997e-06 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.464 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624366+0000", 2024-09-15T01:45:02.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1628000000000003e-05 2024-09-15T01:45:02.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.467 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.624377+0000", 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0227000000000001e-05 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625038+0000", 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00066086900000000002 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:02.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.625186+0000", 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000148152 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636824+0000", 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011638002999999999 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.636959+0000", 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000135387 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.637099+0000", 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014027700000000001 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:541 3.0s0 3:0b545d74:::smithi03327189-8:head [append 0~524288] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.767982178, 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0095842170000000008, 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.470 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 541 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644217+0000", 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644218+0000", 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1990000000000001e-06 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644761+0000", 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00054275099999999995 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644777+0000", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6339000000000001e-05 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644783+0000", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9279999999999998e-06 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644895+0000", 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011205999999999999 2024-09-15T01:45:02.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.644988+0000", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2645999999999994e-05 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645895+0000", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00090718299999999999 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649455+0000", 2024-09-15T01:45:02.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003560026 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653727+0000", 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004272662 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.653801+0000", 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.3423000000000005e-05 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:543 3.0s0 3:0b545d74:::smithi03327189-8:head [setxattr _header (56) in=7b] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.766619515999999, 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0093488740000000001, 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 543 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.476 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645579+0000", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645580+0000", 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2799999999999996e-07 2024-09-15T01:45:02.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645582+0000", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1660000000000001e-06 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645588+0000", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1639999999999998e-06 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645591+0000", 2024-09-15T01:45:02.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8270000000000002e-06 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648370+0000", 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027791370000000001 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.648467+0000", 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6657000000000006e-05 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649053+0000", 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00058605500000000004 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.650538+0000", 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014853469999999999 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654864+0000", 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004325354 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.654928+0000", 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4539e-05 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:544 3.0s0 3:0b545d74:::smithi03327189-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.766604991000001, 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012236567, 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 544 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645594+0000", 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3700000000000001e-07 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645596+0000", 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.544e-06 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645600+0000", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7919999999999999e-06 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.645602+0000", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6630000000000002e-06 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649704+0000", 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041017060000000001 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.649784+0000", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9851999999999994e-05 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655405+0000", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005621526 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.655490+0000", 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4815999999999996e-05 2024-09-15T01:45:02.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.657830+0000", 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0023403310000000002 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:557 3.0s0 3:0899e339:::smithi03327189-38:head [append 0~606208] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.744219229999999, 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010317464, 2024-09-15T01:45:02.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 557 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.487 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667980+0000", 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.667981+0000", 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5060000000000001e-06 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668673+0000", 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069155599999999996 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668687+0000", 2024-09-15T01:45:02.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4759e-05 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668695+0000", 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6170000000000001e-06 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668826+0000", 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013106899999999999 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.668971+0000", 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014515700000000001 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.670379+0000", 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014081790000000001 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.676195+0000", 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058153379999999998 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678203+0000", 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002008583 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.678297+0000", 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3700000000000001e-05 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.492 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:561 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 24.740448908000001, 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010840044, 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 561 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671750+0000", 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671751+0000", 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6199999999999997e-07 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671754+0000", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0759999999999999e-06 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671762+0000", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4679999999999992e-06 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.671767+0000", 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.7350000000000004e-06 2024-09-15T01:45:02.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677604+0000", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0058372770000000001 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.677723+0000", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011872200000000001 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681140+0000", 2024-09-15T01:45:02.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034173699999999999 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.681253+0000", 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000112757 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:37.682590+0000", 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001336877 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.895324066000001, 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.498 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:45:02.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:675 3.cs0 3:3584cd82:::smithi03327189-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.895207026, 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017344149999999999, 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 675 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516992+0000", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516993+0000", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5600000000000001e-06 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517001+0000", 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1489999999999997e-06 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517020+0000", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.9364999999999998e-05 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517028+0000", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.6229999999999997e-06 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517199+0000", 2024-09-15T01:45:02.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017127399999999999 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7309999999999998e-06 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524428+0000", 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072231680000000003 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.524522+0000", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4467000000000002e-05 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534164+0000", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0096419079999999994 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534243+0000", 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.9296999999999997e-05 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534336+0000", 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2608000000000005e-05 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:680 3.0s0 3:0a6b0087:::smithi03327189-4:head [append 0~524288] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.869088733, 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010598392999999999, 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 680 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543110+0000", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.543113+0000", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5490000000000001e-06 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544026+0000", 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00091366600000000004 2024-09-15T01:45:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544053+0000", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7015000000000001e-05 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544068+0000", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4977000000000001e-05 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544122+0000", 2024-09-15T01:45:02.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3804000000000002e-05 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.544197+0000", 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5139e-05 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.545372+0000", 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001175024 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.547186+0000", 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001814026 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553517+0000", 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0063304659999999999 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.553709+0000", 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019172699999999999 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:685 3.0s0 3:0a6b0087:::smithi03327189-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 15.855443068, 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.01007038, 2024-09-15T01:45:02.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 685 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556756+0000", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8299999999999997e-07 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556760+0000", 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7189999999999999e-06 2024-09-15T01:45:02.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556766+0000", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3810000000000004e-06 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.556770+0000", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6109999999999998e-06 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.562875+0000", 2024-09-15T01:45:02.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061050740000000003 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.563061+0000", 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000186066 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564501+0000", 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014400400000000001 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.564606+0000", 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010522399999999999 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.566826+0000", 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002219782 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.521 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:773 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 30] snapc 2d=[2a] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.8932425390000001, 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026472887000000001, 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 773 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518960+0000", 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1769999999999998e-06 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518990+0000", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519067+0000", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7348000000000006e-05 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519085+0000", 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8050000000000002e-05 2024-09-15T01:45:02.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519275+0000", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018974899999999999 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519561+0000", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028586399999999999 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.539158+0000", 2024-09-15T01:45:02.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019596921999999999 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544838+0000", 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056797690000000003 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545308+0000", 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00047005099999999998 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.526 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545429+0000", 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121575 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:774 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.8930803730000001, 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025029231999999998, 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 774 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:02.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519121+0000", 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2330000000000001e-06 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519129+0000", 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7700000000000007e-06 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519153+0000", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3017e-05 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519164+0000", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1953000000000001e-05 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519853+0000", 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068810500000000005 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519993+0000", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037699999999999 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.543813+0000", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023819931999999999 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544004+0000", 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019107799999999999 2024-09-15T01:45:02.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544148+0000", 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014376699999999999 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:02.532 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:02.534 INFO:tasks.rados.rados.0.smithi033.stdout:313: done (2 left) 2024-09-15T01:45:02.534 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 6 current snap is 25 2024-09-15T01:45:02.534 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 1 to smithi03327189-17 2024-09-15T01:45:02.534 INFO:tasks.rados.rados.0.smithi033.stdout:314: seq_num 137 ranges {98304=753664,851968=786432,1638400=655360,2293760=114688} 2024-09-15T01:45:02.534 INFO:tasks.rados.rados.0.smithi033.stdout:314: writing smithi03327189-6 from 98304 to 851968 tid 1 2024-09-15T01:45:02.535 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:02.528+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:02.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:45:02.537 INFO:tasks.rados.rados.0.smithi033.stdout:314: writing smithi03327189-6 from 851968 to 1638400 tid 2 2024-09-15T01:45:02.541 INFO:tasks.rados.rados.0.smithi033.stdout:314: writing smithi03327189-6 from 1638400 to 2293760 tid 3 2024-09-15T01:45:02.542 INFO:tasks.rados.rados.0.smithi033.stdout:314: writing smithi03327189-6 from 2293760 to 2408448 tid 4 2024-09-15T01:45:02.542 INFO:tasks.rados.rados.0.smithi033.stdout:315: setattr oid 45 current snap is 25 2024-09-15T01:45:02.544 INFO:tasks.rados.rados.0.smithi033.stdout:312: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 2 to smithi03327189-17 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 3 to smithi03327189-17 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 4 to smithi03327189-17 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 5 to smithi03327189-17 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: finishing write tid 6 to smithi03327189-17 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 17 v 65 (ObjNum 136 snap 25 seq_num 136) dirty exists 2024-09-15T01:45:02.553 INFO:tasks.rados.rados.0.smithi033.stdout:311: left oid 17 (ObjNum 136 snap 25 seq_num 136) 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:311: done (3 left) 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:312: done (2 left) 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:316: copy_from oid 2 from oid 17 current snap is 25 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:317: read oid 27 snap -1 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:317: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T01:45:02.554 INFO:tasks.rados.rados.0.smithi033.stdout:318: delete oid 28 current snap is 25 2024-09-15T01:45:02.558 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 1 to smithi03327189-6 2024-09-15T01:45:02.560 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 2 to smithi03327189-6 2024-09-15T01:45:02.564 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 3 to smithi03327189-6 2024-09-15T01:45:02.565 INFO:tasks.rados.rados.0.smithi033.stdout:317: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T01:45:02.583 INFO:tasks.rados.rados.0.smithi033.stdout:317: done (4 left) 2024-09-15T01:45:02.583 INFO:tasks.rados.rados.0.smithi033.stdout:318: done (3 left) 2024-09-15T01:45:02.583 INFO:tasks.rados.rados.0.smithi033.stdout:319: delete oid 19 current snap is 25 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 4 to smithi03327189-6 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 5 to smithi03327189-6 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:314: finishing write tid 6 to smithi03327189-6 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 108 (ObjNum 137 snap 25 seq_num 137) dirty exists 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:314: left oid 6 (ObjNum 137 snap 25 seq_num 137) 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 45 v 161 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:316: finishing copy_from racing read to smithi03327189-2 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:316: finishing copy_from to smithi03327189-2 2024-09-15T01:45:02.584 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 2 v 65 (ObjNum 136 snap 25 seq_num 136) dirty exists 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:314: done (3 left) 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:315: done (2 left) 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:316: done (1 left) 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:319: done (0 left) 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:320: rmattr oid 21 current snap is 25 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 26 current snap is 25 2024-09-15T01:45:02.588 INFO:tasks.rados.rados.0.smithi033.stdout:321: seq_num 140 ranges {425984=802816,1228800=425984,1654784=720896,2375680=409600} 2024-09-15T01:45:02.596 INFO:tasks.rados.rados.0.smithi033.stdout:321: writing smithi03327189-26 from 425984 to 1228800 tid 1 2024-09-15T01:45:02.599 INFO:tasks.rados.rados.0.smithi033.stdout:321: writing smithi03327189-26 from 1228800 to 1654784 tid 2 2024-09-15T01:45:02.604 INFO:tasks.rados.rados.0.smithi033.stdout:321: writing smithi03327189-26 from 1654784 to 2375680 tid 3 2024-09-15T01:45:02.607 INFO:tasks.rados.rados.0.smithi033.stdout:321: writing smithi03327189-26 from 2375680 to 2785280 tid 4 2024-09-15T01:45:02.607 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 38 current snap is 25 2024-09-15T01:45:02.607 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 21 v 49 (ObjNum 107 snap 13 seq_num 107) dirty exists 2024-09-15T01:45:02.607 INFO:tasks.rados.rados.0.smithi033.stdout:322: seq_num 141 ranges {1769472=425984,2195456=573440,2768896=425984,3194880=524288,3719168=425984} 2024-09-15T01:45:02.620 INFO:tasks.rados.rados.0.smithi033.stdout:322: writing smithi03327189-38 from 1769472 to 2195456 tid 1 2024-09-15T01:45:02.623 INFO:tasks.rados.rados.0.smithi033.stdout:322: writing smithi03327189-38 from 2195456 to 2768896 tid 2 2024-09-15T01:45:02.625 INFO:tasks.rados.rados.0.smithi033.stdout:322: writing smithi03327189-38 from 2768896 to 3194880 tid 3 2024-09-15T01:45:02.629 INFO:tasks.rados.rados.0.smithi033.stdout:322: writing smithi03327189-38 from 3194880 to 3719168 tid 4 2024-09-15T01:45:02.632 INFO:tasks.rados.rados.0.smithi033.stdout:322: writing smithi03327189-38 from 3719168 to 4145152 tid 5 2024-09-15T01:45:02.632 INFO:tasks.rados.rados.0.smithi033.stdout:320: done (2 left) 2024-09-15T01:45:02.632 INFO:tasks.rados.rados.0.smithi033.stdout:323: copy_from oid 47 from oid 39 current snap is 25 2024-09-15T01:45:02.632 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 1 to smithi03327189-26 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 2 to smithi03327189-26 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 3 to smithi03327189-26 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 4 to smithi03327189-26 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 5 to smithi03327189-26 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:324: read oid 31 snap -1 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:324: expect (ObjNum 104 snap 13 seq_num 104) 2024-09-15T01:45:02.633 INFO:tasks.rados.rados.0.smithi033.stdout:325: delete oid 2 current snap is 25 2024-09-15T01:45:02.634 INFO:tasks.rados.rados.0.smithi033.stdout:321: finishing write tid 6 to smithi03327189-26 2024-09-15T01:45:02.634 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 26 v 166 (ObjNum 140 snap 25 seq_num 140) dirty exists 2024-09-15T01:45:02.634 INFO:tasks.rados.rados.0.smithi033.stdout:321: left oid 26 (ObjNum 140 snap 25 seq_num 140) 2024-09-15T01:45:02.634 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 1 to smithi03327189-38 2024-09-15T01:45:02.634 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 2 to smithi03327189-38 2024-09-15T01:45:02.635 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:02.627+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:02.636 INFO:tasks.rados.rados.0.smithi033.stdout:321: done (4 left) 2024-09-15T01:45:02.636 INFO:tasks.rados.rados.0.smithi033.stdout:325: done (3 left) 2024-09-15T01:45:02.636 INFO:tasks.rados.rados.0.smithi033.stdout:326: copy_from oid 12 from oid 45 current snap is 25 2024-09-15T01:45:02.636 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 4 current snap is 25 2024-09-15T01:45:02.636 INFO:tasks.rados.rados.0.smithi033.stdout:327: seq_num 143 ranges {1720320=458752,2179072=475136,2654208=655360,3309568=671744,3981312=147456} 2024-09-15T01:45:02.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:45:02.649 INFO:tasks.rados.rados.0.smithi033.stdout:327: writing smithi03327189-4 from 1720320 to 2179072 tid 1 2024-09-15T01:45:02.652 INFO:tasks.rados.rados.0.smithi033.stdout:327: writing smithi03327189-4 from 2179072 to 2654208 tid 2 2024-09-15T01:45:02.659 INFO:tasks.rados.rados.0.smithi033.stdout:327: writing smithi03327189-4 from 2654208 to 3309568 tid 3 2024-09-15T01:45:02.659 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:02.660 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:02.663 INFO:tasks.rados.rados.0.smithi033.stdout:327: writing smithi03327189-4 from 3309568 to 3981312 tid 4 2024-09-15T01:45:02.665 INFO:tasks.rados.rados.0.smithi033.stdout:327: writing smithi03327189-4 from 3981312 to 4128768 tid 5 2024-09-15T01:45:02.665 INFO:tasks.rados.rados.0.smithi033.stdout:328: copy_from oid 18 from oid 35 current snap is 25 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 3 to smithi03327189-38 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 4 to smithi03327189-38 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 5 to smithi03327189-38 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 6 to smithi03327189-38 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: finishing write tid 7 to smithi03327189-38 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 114 (ObjNum 141 snap 25 seq_num 141) dirty exists 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: left oid 38 (ObjNum 141 snap 25 seq_num 141) 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:322: done (5 left) 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:329: snap_create 2024-09-15T01:45:02.666 INFO:tasks.rados.rados.0.smithi033.stdout:324: expect (ObjNum 104 snap 13 seq_num 104) 2024-09-15T01:45:02.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:45:02.735 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:02.727+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:02.770 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:02.788 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 1 to smithi03327189-4 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:323: finishing copy_from racing read to smithi03327189-47 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 2 to smithi03327189-4 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:326: finishing copy_from racing read to smithi03327189-12 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 3 to smithi03327189-4 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:323: finishing copy_from to smithi03327189-47 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 47 v 52 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:326: finishing copy_from to smithi03327189-12 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 161 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 4 to smithi03327189-4 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 5 to smithi03327189-4 2024-09-15T01:45:02.795 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 6 to smithi03327189-4 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:327: finishing write tid 7 to smithi03327189-4 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 120 (ObjNum 143 snap 25 seq_num 143) dirty exists 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:327: left oid 4 (ObjNum 143 snap 25 seq_num 143) 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:328: finishing copy_from racing read to smithi03327189-18 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:328: finishing copy_from to smithi03327189-18 2024-09-15T01:45:02.796 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 38 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:45:02.807 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:02.824 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:02.824 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:02.824 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:02.824 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:02.824 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:02.827 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:02.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:45:02.836 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:02.831+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:02.851 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:02.879 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:02.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:02.930 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:02.930 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:02.936 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:02.931+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:02.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:45:02.941 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:02.964 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:02.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:03.013 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:03.042 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.042 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:45:03.072 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:03.068+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:03.138 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:03.132+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:03.154 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.154 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:45:03.182 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:45:03.182 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:45:03.182 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-15T01:45:03.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-15T01:45:03.238 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:03.235+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:03.259 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.259 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:45:03.339 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:03.332+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:03.374 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.374 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:45:03.441 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:03.439+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:03.482 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.483 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:45:03.493 INFO:teuthology.orchestra.run.smithi033.stdout:moved 0 / 146 (0%) 2024-09-15T01:45:03.493 INFO:teuthology.orchestra.run.smithi033.stdout:avg 14.6 2024-09-15T01:45:03.493 INFO:teuthology.orchestra.run.smithi033.stdout:stddev 1.42829 -> 1.42829 (expected baseline 3.62491) 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:min osd.2 with 12 -> 12 pgs (0.821918 -> 0.821918 * mean) 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:max osd.1 with 17 -> 17 pgs (1.16438 -> 1.16438 * mean) 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:oload 1000 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:max_change 0.05 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:max_change_osds 4 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:average_utilization 0.0034 2024-09-15T01:45:03.494 INFO:teuthology.orchestra.run.smithi033.stdout:overload_utilization 0.0342 2024-09-15T01:45:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:no change 2024-09-15T01:45:03.507 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-15T01:45:03.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-15T01:45:03.541 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:03.539+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:03.570 INFO:tasks.rados.rados.0.smithi033.stdout:323: done (5 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:324: done (4 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:326: done (3 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:327: done (2 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:328: done (1 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:329: done (0 left) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:330: read oid 41 snap -1 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:330: expect (ObjNum 82 snap 8 seq_num 82) 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 14 current snap is 26 2024-09-15T01:45:03.571 INFO:tasks.rados.rados.0.smithi033.stdout:331: seq_num 144 ranges {4079616=16384} 2024-09-15T01:45:03.599 INFO:tasks.rados.rados.0.smithi033.stdout:331: writing smithi03327189-14 from 4079616 to 4096000 tid 1 2024-09-15T01:45:03.600 INFO:tasks.rados.rados.0.smithi033.stdout:332: setattr oid 20 current snap is 26 2024-09-15T01:45:03.600 INFO:tasks.rados.rados.0.smithi033.stdout:330: expect (ObjNum 82 snap 8 seq_num 82) 2024-09-15T01:45:03.603 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.603 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:45:03.614 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:45:03.642 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:03.639+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:03.714 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.714 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:45:03.743 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:03.736+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:03.781 INFO:tasks.rados.rados.0.smithi033.stdout:331: finishing write tid 1 to smithi03327189-14 2024-09-15T01:45:03.781 INFO:tasks.rados.rados.0.smithi033.stdout:331: finishing write tid 2 to smithi03327189-14 2024-09-15T01:45:03.781 INFO:tasks.rados.rados.0.smithi033.stdout:331: finishing write tid 3 to smithi03327189-14 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 71 (ObjNum 144 snap 26 seq_num 144) dirty exists 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:331: left oid 14 (ObjNum 144 snap 26 seq_num 144) 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:330: done (2 left) 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:331: done (1 left) 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:333: read oid 41 snap -1 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:333: expect (ObjNum 82 snap 8 seq_num 82) 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:334: copy_from oid 49 from oid 35 current snap is 26 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:335: copy_from oid 44 from oid 16 current snap is 26 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:336: copy_from oid 40 from oid 14 current snap is 26 2024-09-15T01:45:03.782 INFO:tasks.rados.rados.0.smithi033.stdout:337: snap_create 2024-09-15T01:45:03.789 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 20 v 64 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:moved 0 / 146 (0%) 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:avg 14.6 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:stddev 1.42829 -> 1.42829 (expected baseline 3.62491) 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:min osd.2 with 12 -> 12 pgs (0.821918 -> 0.821918 * mean) 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:max osd.1 with 17 -> 17 pgs (1.16438 -> 1.16438 * mean) 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:oload 110 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:max_change 0.05 2024-09-15T01:45:03.795 INFO:teuthology.orchestra.run.smithi033.stdout:max_change_osds 4 2024-09-15T01:45:03.796 INFO:teuthology.orchestra.run.smithi033.stdout:average_utilization 0.0034 2024-09-15T01:45:03.796 INFO:teuthology.orchestra.run.smithi033.stdout:overload_utilization 0.0038 2024-09-15T01:45:03.799 INFO:teuthology.orchestra.run.smithi033.stderr:no change 2024-09-15T01:45:03.811 INFO:tasks.rados.rados.0.smithi033.stdout:336: finishing copy_from racing read to smithi03327189-40 2024-09-15T01:45:03.815 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-15T01:45:03.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-15T01:45:03.818 INFO:tasks.rados.rados.0.smithi033.stdout:333: expect (ObjNum 82 snap 8 seq_num 82) 2024-09-15T01:45:03.824 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.824 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:45:03.844 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:03.840+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:03.924 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:03.925 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:03.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:45:03.945 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:03.940+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:336: finishing copy_from to smithi03327189-40 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 40 v 71 (ObjNum 144 snap 26 seq_num 144) dirty exists 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:334: finishing copy_from racing read to smithi03327189-49 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:335: finishing copy_from racing read to smithi03327189-44 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:334: finishing copy_from to smithi03327189-49 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 49 v 38 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:335: finishing copy_from to smithi03327189-44 2024-09-15T01:45:03.994 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 44 v 16 (ObjNum 127 snap 20 seq_num 127) dirty exists 2024-09-15T01:45:04.031 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.031 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:45:04.046 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:04.040+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:moved 0 / 146 (0%) 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:avg 14.6 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:stddev 1.42829 -> 1.42829 (expected baseline 3.62491) 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:min osd.2 with 12 -> 12 pgs (0.821918 -> 0.821918 * mean) 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:max osd.1 with 17 -> 17 pgs (1.16438 -> 1.16438 * mean) 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:04.120 INFO:teuthology.orchestra.run.smithi033.stdout:oload 1000 2024-09-15T01:45:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:max_change 0.05 2024-09-15T01:45:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:max_change_osds 4 2024-09-15T01:45:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:average_utilization 0.0034 2024-09-15T01:45:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:overload_utilization 0.0342 2024-09-15T01:45:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:no change 2024-09-15T01:45:04.134 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-15T01:45:04.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-15T01:45:04.139 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.139 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.148 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:45:04.176 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:04.172+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:04.248 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:04.244+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:04.250 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.250 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.259 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:45:04.349 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:04.347+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:04.362 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.362 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.372 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:45:04.451 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:04.443+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:04.470 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.470 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout:moved 0 / 146 (0%) 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout:avg 14.6 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout:stddev 1.42829 -> 1.42829 (expected baseline 3.62491) 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout:min osd.2 with 12 -> 12 pgs (0.821918 -> 0.821918 * mean) 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout:max osd.1 with 17 -> 17 pgs (1.16438 -> 1.16438 * mean) 2024-09-15T01:45:04.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:oload 110 2024-09-15T01:45:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:max_change 1 2024-09-15T01:45:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:max_change_osds 4 2024-09-15T01:45:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:average_utilization 0.0034 2024-09-15T01:45:04.483 INFO:teuthology.orchestra.run.smithi033.stdout:overload_utilization 0.0038 2024-09-15T01:45:04.486 INFO:teuthology.orchestra.run.smithi033.stderr:no change 2024-09-15T01:45:04.497 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-15T01:45:04.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-15T01:45:04.552 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:04.548+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:332: done (5 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:333: done (4 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:334: done (3 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:335: done (2 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:336: done (1 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:337: done (0 left) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:338: read oid 16 snap 26 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:338: expect (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:45:04.579 INFO:tasks.rados.rados.0.smithi033.stdout:339: delete oid 40 current snap is 27 2024-09-15T01:45:04.585 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:45:04.595 INFO:tasks.rados.rados.0.smithi033.stdout:338: expect (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:45:04.628 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.629 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.638 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:45:04.653 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:04.648+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:04.671 INFO:tasks.rados.rados.0.smithi033.stdout:338: done (1 left) 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:339: done (0 left) 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:340: read oid 13 snap -1 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:340: expect (ObjNum 99 snap 13 seq_num 99) 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:341: rmattr oid 16 current snap is 27 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:342: read oid 14 snap 26 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:342: expect (ObjNum 144 snap 26 seq_num 144) 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:343: read oid 44 snap -1 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:343: expect (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:45:04.672 INFO:tasks.rados.rados.0.smithi033.stdout:344: setattr oid 38 current snap is 27 2024-09-15T01:45:04.675 INFO:tasks.rados.rados.0.smithi033.stdout:345: setattr oid 34 current snap is 27 2024-09-15T01:45:04.682 INFO:tasks.rados.rados.0.smithi033.stdout:346: snap_create 2024-09-15T01:45:04.685 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 16 v 73 (ObjNum 127 snap 20 seq_num 127) dirty exists 2024-09-15T01:45:04.689 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 122 (ObjNum 141 snap 25 seq_num 141) dirty exists 2024-09-15T01:45:04.693 INFO:tasks.rados.rados.0.smithi033.stdout:343: expect (ObjNum 127 snap 20 seq_num 127) 2024-09-15T01:45:04.741 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.741 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:45:04.753 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:04.747+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:04.765 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 34 v 124 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T01:45:04.766 INFO:tasks.rados.rados.0.smithi033.stdout:340: expect (ObjNum 99 snap 13 seq_num 99) 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout:moved 0 / 146 (0%) 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout:avg 14.6 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout:stddev 1.42829 -> 1.42829 (expected baseline 3.62491) 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout:min osd.2 with 12 -> 12 pgs (0.821918 -> 0.821918 * mean) 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout:max osd.1 with 17 -> 17 pgs (1.16438 -> 1.16438 * mean) 2024-09-15T01:45:04.792 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:04.793 INFO:teuthology.orchestra.run.smithi033.stdout:oload 1000 2024-09-15T01:45:04.793 INFO:teuthology.orchestra.run.smithi033.stdout:max_change 1 2024-09-15T01:45:04.793 INFO:teuthology.orchestra.run.smithi033.stdout:max_change_osds 4 2024-09-15T01:45:04.793 INFO:teuthology.orchestra.run.smithi033.stdout:average_utilization 0.0034 2024-09-15T01:45:04.793 INFO:teuthology.orchestra.run.smithi033.stdout:overload_utilization 0.0342 2024-09-15T01:45:04.795 INFO:teuthology.orchestra.run.smithi033.stderr:no change 2024-09-15T01:45:04.814 INFO:tasks.rados.rados.0.smithi033.stdout:342: expect (ObjNum 144 snap 26 seq_num 144) 2024-09-15T01:45:04.847 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:04.847 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:04.854 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:04.847+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:04.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:45:04.955 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:04.947+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:05.003 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:05.003 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:05.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:45:05.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:45:05.072 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:05.068+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:05.109 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:05.109 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:05.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:45:05.157 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:05.155+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:05.162 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:05.189 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:05.208 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:05.208 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:05.210 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:05.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:45:05.232 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:05.254 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:05.258 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:05.255+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:05.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:05.298 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:05.320 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:05.320 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:05.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:05.320 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.788502989000001, 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.321 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:45:05.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.786071591000002, 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:45:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:45:05.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.785832775999999, 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:45:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:45:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:45:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.785801393, 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:45:05.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.806593962999997, 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:45:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.798679561, 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:45:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:45:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:45:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:45:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.798650297999998, 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:45:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.799074978999997, 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:45:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:45:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:45:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.796836517999999, 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:45:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:45:05.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:45:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.796266848999998, 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.376 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:45:05.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.382 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.796207881000001, 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:45:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.808309672999997, 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:45:05.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:45:05.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.808215895000004, 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:45:05.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:45:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:45:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:45:05.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:493 3.6s0 3:6579c50e:::smithi03327189-13:head [append 0~507904] snapc 19=[19,18] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.741571781000001, 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551495, 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 493 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573379+0000", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573381+0000", 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6879999999999999e-06 2024-09-15T01:45:05.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573780+0000", 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039826400000000001 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573824+0000", 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4045999999999998e-05 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573838+0000", 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.413e-05 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.573922+0000", 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4442000000000002e-05 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.574104+0000", 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018210500000000001 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.575895+0000", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017907579999999999 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.578241+0000", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023458569999999998 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584820+0000", 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065789300000000002 2024-09-15T01:45:05.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.584930+0000", 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110275 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=117, reqid=osd.4.0:117, at_version=68'74, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.800480503999999, 2024-09-15T01:45:05.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011721416, 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514471+0000", 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514541+0000", 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0955000000000001e-05 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514567+0000", 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.597e-05 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514584+0000", 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6463000000000001e-05 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514727+0000", 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014272899999999999 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514754+0000", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7290000000000001e-05 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526159+0000", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011404621 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526191+0000", 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2840999999999999e-05 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/53 ECSubWrite(tid=118, reqid=osd.4.0:118, at_version=68'50, trim_to=0'0, roll_forward_to=67'48))", 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.800073255000001, 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012184878999999999, 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:05.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514879+0000", 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514938+0000", 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9142999999999999e-05 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514963+0000", 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5165999999999999e-05 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515032+0000", 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8602999999999994e-05 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515160+0000", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012877200000000001 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515188+0000", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7313000000000001e-05 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527007+0000", 2024-09-15T01:45:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011819079999999999 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527062+0000", 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5328999999999998e-05 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=68'76, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.798018872, 2024-09-15T01:45:05.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014580965, 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516933+0000", 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6249999999999999e-06 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516977+0000", 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3551999999999999e-05 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517002+0000", 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4762000000000001e-05 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517016+0000", 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.401e-05 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517789+0000", 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077353199999999995 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517813+0000", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3748e-05 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531475+0000", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013662636000000001 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531513+0000", 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7100000000000001e-05 2024-09-15T01:45:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:753 3.1s0 3:86165cc9:::smithi03327189-11:head [copy-from ver 13] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.713598892, 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022114891000000001, 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 753 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601354+0000", 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6000000000000001e-06 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601366+0000", 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601396+0000", 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9901999999999999e-05 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601406+0000", 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601519+0000", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112546 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601677+0000", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015800100000000001 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.614051+0000", 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012374232000000001 2024-09-15T01:45:05.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617994+0000", 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039432089999999996 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623415+0000", 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054204240000000001 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623466+0000", 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1972000000000003e-05 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:754 3.1s0 3:86165cc9:::smithi03327189-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.713519245000001, 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015926781000000001, 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 754 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601433+0000", 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4270000000000001e-06 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601437+0000", 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2899999999999996e-06 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601452+0000", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4939999999999999e-05 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601461+0000", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601979+0000", 2024-09-15T01:45:05.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051790899999999999 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.602057+0000", 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8237000000000003e-05 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617204+0000", 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015147361999999999 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617280+0000", 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6318999999999999e-05 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617358+0000", 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7570000000000004e-05 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:6 3.6s0 3:6579c50e:::smithi03327189-13:head [assert-version v30,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:05.432 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 7.7951930789999997, 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011523049, 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519757+0000", 2024-09-15T01:45:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519761+0000", 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519774+0000", 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3550000000000001e-05 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519826+0000", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.519847+0000", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0890000000000002e-05 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.520007+0000", 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000159801 2024-09-15T01:45:05.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.520028+0000", 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1696999999999999e-05 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.525500+0000", 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054716089999999997 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.525646+0000", 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014582700000000001 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:57.531280+0000", 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056347089999999999 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.437 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:05.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:05.439 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:05.431+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:05.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:45:05.441 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:05.441 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:05.441 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:05.441 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:05.442 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:05.443 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:05.540 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:05.535+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:05.590 INFO:tasks.rados.rados.0.smithi033.stdout:340: done (6 left) 2024-09-15T01:45:05.591 INFO:tasks.rados.rados.0.smithi033.stdout:341: done (5 left) 2024-09-15T01:45:05.591 INFO:tasks.rados.rados.0.smithi033.stdout:342: done (4 left) 2024-09-15T01:45:05.591 INFO:tasks.rados.rados.0.smithi033.stdout:343: done (3 left) 2024-09-15T01:45:05.591 INFO:tasks.rados.rados.0.smithi033.stdout:344: done (2 left) 2024-09-15T01:45:05.591 INFO:tasks.rados.rados.0.smithi033.stdout:345: done (1 left) 2024-09-15T01:45:05.592 INFO:tasks.rados.rados.0.smithi033.stdout:346: done (0 left) 2024-09-15T01:45:05.592 INFO:tasks.rados.rados.0.smithi033.stdout:347: copy_from oid 12 from oid 21 current snap is 28 2024-09-15T01:45:05.592 INFO:tasks.rados.rados.0.smithi033.stdout:348: snap_create 2024-09-15T01:45:05.598 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:05.598 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:05.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:45:05.614 INFO:tasks.rados.rados.0.smithi033.stdout:347: finishing copy_from racing read to smithi03327189-12 2024-09-15T01:45:05.619 INFO:tasks.rados.rados.0.smithi033.stdout:347: finishing copy_from to smithi03327189-12 2024-09-15T01:45:05.619 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 49 (ObjNum 107 snap 13 seq_num 107) dirty exists 2024-09-15T01:45:05.641 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:05.636+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:05.702 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:05.703 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:05.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:45:05.743 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:05.739+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:05.844 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:05.839+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:05.862 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.336538337, 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:45:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:45:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.336507392999998, 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:45:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:45:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.339627571999998, 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:45:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:45:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:45:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.33955804, 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:45:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:45:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.339535503999997, 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:45:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:45:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:45:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.339468312000001, 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:45:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:45:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.339398787, 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:45:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:45:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.339156709000001, 2024-09-15T01:45:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:45:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:45:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.339103274999999, 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:45:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:45:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.339031095999999, 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:45:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.333180417000001, 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:45:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:45:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:45:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.328263594999999, 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 73/53 ECSubWrite(tid=132, reqid=osd.4.0:132, at_version=73'84, trim_to=0'0, roll_forward_to=72'82))", 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.340677626, 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015282341, 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:45:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516946+0000", 2024-09-15T01:45:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9145999999999999e-05 2024-09-15T01:45:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516976+0000", 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0236e-05 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516993+0000", 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517168+0000", 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017460800000000001 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517197+0000", 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8813e-05 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532115+0000", 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014917629 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532168+0000", 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3596e-05 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:740 3.4s0 3:20577364:::smithi03327189-24:head [copy-from ver 0] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.337994324, 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016305848000000001, 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 740 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519573+0000", 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-06 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519590+0000", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519639+0000", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9432999999999999e-05 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519658+0000", 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9083e-05 2024-09-15T01:45:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521437+0000", 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017789539999999999 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521452+0000", 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4952e-05 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530454+0000", 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.009001963 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530578+0000", 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012395599999999999 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535875+0000", 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052969169999999999 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:741 3.4s0 3:20577364:::smithi03327189-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.337817036000001, 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016411578999999999, 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 741 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519748+0000", 2024-09-15T01:45:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.04e-06 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519758+0000", 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5929999999999995e-06 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519795+0000", 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6860000000000003e-05 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519810+0000", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5082999999999999e-05 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521558+0000", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017476 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521568+0000", 2024-09-15T01:45:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0344e-05 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530788+0000", 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092198539999999995 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530875+0000", 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7698e-05 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535967+0000", 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918409999999997 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536066+0000", 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8281000000000003e-05 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536158+0000", 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2385000000000004e-05 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=183, reqid=client.4632.0:738, at_version=73'21, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.334023593, 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018357772000000001, 2024-09-15T01:45:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523541+0000", 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.24e-06 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523599+0000", 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7500999999999997e-05 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523628+0000", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9193999999999999e-05 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523640+0000", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2276000000000001e-05 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531608+0000", 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079682079999999992 2024-09-15T01:45:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531619+0000", 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0293e-05 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541866+0000", 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010247114999999999 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541898+0000", 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1945e-05 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 73/34 ECSubWrite(tid=58, reqid=osd.2.0:58, at_version=73'43, trim_to=0'0, roll_forward_to=72'41))", 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.331284269999999, 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015869550999999999, 2024-09-15T01:45:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526280+0000", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999996e-07 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526306+0000", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6282e-05 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526322+0000", 2024-09-15T01:45:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5977999999999998e-05 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526330+0000", 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.191e-06 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526451+0000", 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120526 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526459+0000", 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542088+0000", 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015629295000000001 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542149+0000", 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0247000000000001e-05 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.330090826999999, 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014773232000000001, 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527473+0000", 2024-09-15T01:45:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527518+0000", 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4802000000000002e-05 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527528+0000", 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8640000000000008e-06 2024-09-15T01:45:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527534+0000", 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7590000000000003e-06 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534080+0000", 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065466159999999999 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.534092+0000", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1953999999999999e-05 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542223+0000", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081309510000000008 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542246+0000", 2024-09-15T01:45:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2776999999999999e-05 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:811 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~737280] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3291807460000005, 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022983506000000001, 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 811 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528386+0000", 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.292e-06 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529048+0000", 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066185599999999995 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529087+0000", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9653999999999997e-05 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529102+0000", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4931e-05 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529205+0000", 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103047 2024-09-15T01:45:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529215+0000", 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7159999999999998e-06 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533610+0000", 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043948049999999999 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533964+0000", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000354235 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.536912+0000", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029481400000000001 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551101+0000", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014188383000000001 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551169+0000", 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8662000000000001e-05 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551366+0000", 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019678499999999999 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:812 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~770048] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3238977460000001, 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022084530000000002, 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 812 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533668+0000", 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5969999999999999e-06 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534580+0000", 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091206999999999996 2024-09-15T01:45:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534606+0000", 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5573999999999999e-05 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534627+0000", 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1571000000000001e-05 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541493+0000", 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068657910000000004 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541606+0000", 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011307 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.543077+0000", 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014709670000000001 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.553529+0000", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010451313 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555682+0000", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021530949999999998 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555750+0000", 2024-09-15T01:45:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8481999999999999e-05 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:05.994 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:05.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:45:05.996 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:05.991+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:06.097 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:06.092+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:06.101 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:06.102 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:45:06.198 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:06.191+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:06.257 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:06.257 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:45:06.299 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:06.295+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.846232780999998, 2024-09-15T01:45:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:45:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 57/53 ECSubWrite(tid=19, reqid=client.4632.0:476, at_version=57'77, trim_to=0'0, roll_forward_to=56'75))", 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.838729528999998, 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088079779999999993, 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529246+0000", 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529247+0000", 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4799999999999997e-07 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529486+0000", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023889000000000001 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529504+0000", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8678e-05 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.529522+0000", 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7342000000000001e-05 2024-09-15T01:45:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530159+0000", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063701600000000001 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.530177+0000", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8508e-05 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538024+0000", 2024-09-15T01:45:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078470180000000007 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.538054+0000", 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9677999999999999e-05 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.718169157999998, 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:45:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:550 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1720320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.722677108999999, 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010351048, 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 550 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645298+0000", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645299+0000", 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4999999999999998e-07 2024-09-15T01:45:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645301+0000", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645306+0000", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.645309+0000", 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6139999999999999e-06 2024-09-15T01:45:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647705+0000", 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023957940000000001 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.647815+0000", 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011058 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.655649+0000", 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078340649999999994 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.617205033000001, 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:45:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.613765459, 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:45:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:567 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.608979518000002, 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085895820000000001, 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 567 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758996+0000", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.758997+0000", 2024-09-15T01:45:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759002+0000", 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2050000000000001e-06 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759023+0000", 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1070999999999999e-05 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.759033+0000", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6889999999999999e-06 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765216+0000", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061833180000000001 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.765284+0000", 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8085999999999998e-05 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766278+0000", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00099371000000000008 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.766383+0000", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105023 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.767585+0000", 2024-09-15T01:45:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012024150000000001 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.850821901, 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:45:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:45:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.850770981, 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:45:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.850705971, 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:45:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 66/34 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.838751577, 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0088343680000000004, 2024-09-15T01:45:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529224+0000", 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.529227+0000", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.574e-06 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532407+0000", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031793590000000001 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532420+0000", 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3589e-05 2024-09-15T01:45:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532427+0000", 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4910000000000002e-06 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532620+0000", 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193246 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538058+0000", 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005438109 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.835533363, 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:45:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.831489297000001, 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.fs3 76/34 ECSubWrite(tid=192, reqid=client.4632.0:766, at_version=76'30, trim_to=0'0, roll_forward_to=55'28))", 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.775800169, 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091339669999999998, 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592175+0000", 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.592177+0000", 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7600000000000001e-06 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596007+0000", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038296760000000002 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596032+0000", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978e-05 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596042+0000", 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0582e-05 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596154+0000", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111628 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.596174+0000", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0264000000000001e-05 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601267+0000", 2024-09-15T01:45:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050930120000000001 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.601309+0000", 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2067000000000002e-05 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:817 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8278917660000005, 2024-09-15T01:45:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012084411, 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 817 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540086+0000", 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1380000000000001e-06 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540094+0000", 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5210000000000002e-06 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540112+0000", 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7937000000000001e-05 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540121+0000", 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0319999999999992e-06 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540573+0000", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045140399999999998 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540597+0000", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4651000000000002e-05 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546537+0000", 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059398979999999999 2024-09-15T01:45:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546648+0000", 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110967 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552168+0000", 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055198629999999999 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:818 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8278344899999999, 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012454458, 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 818 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540142+0000", 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540146+0000", 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0489999999999997e-06 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540242+0000", 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6168999999999994e-05 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540256+0000", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4063000000000001e-05 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540737+0000", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048110699999999999 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540764+0000", 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7509999999999999e-05 2024-09-15T01:45:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550905+0000", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010140697000000001 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550984+0000", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8906000000000007e-05 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552595+0000", 2024-09-15T01:45:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016113309999999999 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:819 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8277004400000001, 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012574444000000001, 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 819 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540276+0000", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-06 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540284+0000", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.605e-06 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540297+0000", 2024-09-15T01:45:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3526000000000001e-05 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540304+0000", 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2249999999999997e-06 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541504+0000", 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012007179999999999 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541512+0000", 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1099999999999997e-06 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551300+0000", 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097880039999999995 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551387+0000", 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7618e-05 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552849+0000", 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001462281 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:830 3.as0 3:5045eccb:::smithi03327189-27:head [read 0~442368,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.552035+0000", 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.8159402920000001, 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0099890350000000003, 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 830 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552035+0000", 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552035+0000", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552038+0000", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.413e-06 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552046+0000", 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4540000000000007e-06 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552065+0000", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8984999999999999e-05 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552073+0000", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9049999999999997e-06 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552160+0000", 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6614999999999997e-05 2024-09-15T01:45:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552169+0000", 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1349999999999998e-06 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.556730+0000", 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045609819999999999 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.556859+0000", 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012980700000000001 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562024+0000", 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051647389999999998 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:831 3.as0 3:5045eccb:::smithi03327189-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.552208+0000", 2024-09-15T01:45:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.8157669809999999, 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010244069, 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 831 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552208+0000", 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552208+0000", 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552210+0000", 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.065e-06 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552216+0000", 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8499999999999996e-06 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552229+0000", 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2883e-05 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552236+0000", 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7700000000000004e-06 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552474+0000", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023811899999999999 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552481+0000", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9970000000000001e-06 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557203+0000", 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047221800000000003 2024-09-15T01:45:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557256+0000", 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3162999999999999e-05 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562453+0000", 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051960419999999997 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:832 3.as0 3:5045eccb:::smithi03327189-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.8156284199999999, 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010647103, 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 832 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552348+0000", 2024-09-15T01:45:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2079999999999999e-06 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552356+0000", 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9300000000000003e-06 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552373+0000", 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6677999999999999e-05 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552381+0000", 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552513+0000", 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013210000000000001 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552519+0000", 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9020000000000004e-06 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557555+0000", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050360889999999997 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557647+0000", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2503000000000005e-05 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562994+0000", 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053469040000000004 2024-09-15T01:45:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.473 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:45:06.475 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:06.471+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:06.523 INFO:tasks.rados.rados.0.smithi033.stdout:347: done (1 left) 2024-09-15T01:45:06.523 INFO:tasks.rados.rados.0.smithi033.stdout:348: done (0 left) 2024-09-15T01:45:06.523 INFO:tasks.rados.rados.0.smithi033.stdout:349: setattr oid 18 current snap is 29 2024-09-15T01:45:06.524 INFO:tasks.rados.rados.0.smithi033.stdout:350: read oid 42 snap 28 2024-09-15T01:45:06.524 INFO:tasks.rados.rados.0.smithi033.stdout:350: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:45:06.525 INFO:tasks.rados.rados.0.smithi033.stdout:351: read oid 43 snap -1 2024-09-15T01:45:06.525 INFO:tasks.rados.rados.0.smithi033.stdout:351: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:45:06.525 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 14 current snap is 29 2024-09-15T01:45:06.525 INFO:tasks.rados.rados.0.smithi033.stdout:352: seq_num 146 ranges {4096000=49152} 2024-09-15T01:45:06.553 INFO:tasks.rados.rados.0.smithi033.stdout:352: writing smithi03327189-14 from 4096000 to 4145152 tid 1 2024-09-15T01:45:06.553 INFO:tasks.rados.rados.0.smithi033.stdout:353: delete oid 26 current snap is 29 2024-09-15T01:45:06.553 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 170 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:45:06.553 INFO:tasks.rados.rados.0.smithi033.stdout:351: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T01:45:06.570 INFO:tasks.rados.rados.0.smithi033.stdout:350: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T01:45:06.575 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:06.568+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:06.589 INFO:tasks.rados.rados.0.smithi033.stdout:349: done (4 left) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:350: done (3 left) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:351: done (2 left) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:353: done (1 left) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:354: copy_from oid 22 from oid 11 current snap is 29 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:352: finishing write tid 1 to smithi03327189-14 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:352: finishing write tid 2 to smithi03327189-14 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:352: finishing write tid 3 to smithi03327189-14 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 14 v 74 (ObjNum 146 snap 29 seq_num 146) dirty exists 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:352: left oid 14 (ObjNum 146 snap 29 seq_num 146) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:352: done (1 left) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:355: read oid 49 snap -1 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:355: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:356: copy_from oid 28 from oid 20 current snap is 29 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:357: read oid 35 snap -1 2024-09-15T01:45:06.590 INFO:tasks.rados.rados.0.smithi033.stdout:357: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:06.591 INFO:tasks.rados.rados.0.smithi033.stdout:358: copy_from oid 38 from oid 21 current snap is 29 2024-09-15T01:45:06.591 INFO:tasks.rados.rados.0.smithi033.stdout:359: snap_create 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:10 3.2s0 3:47035a48:::smithi03327189-20:head [assert-version v64,copy-get max 8388608 in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.588729+0000", 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0030315059999999998, 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030724020000000001, 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588729+0000", 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588729+0000", 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588731+0000", 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588740+0000", 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.115e-06 2024-09-15T01:45:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588767+0000", 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6485000000000001e-05 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588776+0000", 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.081e-06 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.589877+0000", 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011014169999999999 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590040+0000", 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016297300000000001 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(osd.4.17:11 3.bs0 3:d32e3b1b:::smithi03327189-21:head [assert-version v49,copy-get max 8388608 in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.588871+0000", 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0028896120000000002, 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030031599999999999, 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588871+0000", 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588871+0000", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588872+0000", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5799999999999998e-07 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588884+0000", 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2328e-05 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588895+0000", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1182999999999999e-05 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588901+0000", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.643e-06 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588963+0000", 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2191000000000001e-05 2024-09-15T01:45:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.589056+0000", 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2608999999999994e-05 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpRead(3.2s0 87/53 ECSubRead(tid=252, to_read={3:47035a48:::smithi03327189-20:head=0,421888,0}, subchunks={3:47035a48:::smithi03327189-20:head=[0,1]}, attrs_to_read=))", 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.590489+0000", 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0012713119999999999, 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014548230000000001, 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590489+0000", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590489+0000", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590489+0000", 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590489+0000", 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591424+0000", 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093460099999999996 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591431+0000", 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4100000000000002e-06 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591934+0000", 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050276400000000005 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.bs0 87/34 ECSubReadReply(tid=251, attrs_read=0))", 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.591768+0000", 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4294967295.9999924, 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023385800000000001, 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591768+0000", 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591768+0000", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591768+0000", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591768+0000", 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591792+0000", 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3648000000000001e-05 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591796+0000", 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8720000000000001e-06 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.bs0 87/34 ECSubReadReply(tid=251, attrs_read=0))", 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.590771+0000", 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00098916600000000009, 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001279718, 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590771+0000", 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590771+0000", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.590772+0000", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8800000000000001e-07 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591939+0000", 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001166683 2024-09-15T01:45:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591952+0000", 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3023000000000001e-05 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.591968+0000", 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6336000000000001e-05 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 5 2024-09-15T01:45:06.612 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T01:45:06.613 INFO:tasks.rados.rados.0.smithi033.stdout:354: finishing copy_from racing read to smithi03327189-22 2024-09-15T01:45:06.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:45:06.614 INFO:tasks.rados.rados.0.smithi033.stdout:354: finishing copy_from to smithi03327189-22 2024-09-15T01:45:06.614 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 22 v 13 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:45:06.617 INFO:tasks.rados.rados.0.smithi033.stdout:358: finishing copy_from racing read to smithi03327189-38 2024-09-15T01:45:06.617 INFO:tasks.rados.rados.0.smithi033.stdout:355: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:06.676 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:06.671+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:06.715 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:06.715 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:45:06.777 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:06.774+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:06.796 INFO:tasks.rados.rados.0.smithi033.stdout:357: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.316257163000003, 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.307282262999998, 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.307214600999998, 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:45:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.307188859999997, 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:45:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:45:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.201918935999998, 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:45:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:45:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.222610325000002, 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:45:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:45:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.22247686, 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:45:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:45:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.301785580000001, 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:45:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:45:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.298154149, 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:45:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:45:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.298080474000001, 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:45:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 73/66 ECSubWrite(tid=59, reqid=osd.2.0:59, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.298990236, 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011892049999999999, 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5900000000000002e-07 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526791+0000", 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1916999999999998e-05 2024-09-15T01:45:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526801+0000", 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7759999999999992e-06 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526805+0000", 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2570000000000001e-06 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527105+0000", 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029965900000000003 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527122+0000", 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6993000000000001e-05 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538613+0000", 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011490879000000001 2024-09-15T01:45:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538631+0000", 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8110000000000001e-05 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.29735724, 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016700869, 2024-09-15T01:45:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2e-07 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528400+0000", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7503000000000001e-05 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528410+0000", 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0137e-05 2024-09-15T01:45:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528415+0000", 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4240000000000001e-06 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535553+0000", 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071381459999999997 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535559+0000", 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5210000000000002e-06 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545040+0000", 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094807480000000006 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545073+0000", 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969999999999998e-05 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 76/53 ECSubWrite(tid=189, reqid=osd.7.0:189, at_version=76'28, trim_to=0'0, roll_forward_to=75'26))", 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.296990952, 2024-09-15T01:45:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013688799999999999, 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528759+0000", 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1240999999999999e-05 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528776+0000", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6166000000000001e-05 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528783+0000", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528831+0000", 2024-09-15T01:45:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7750000000000002e-05 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528838+0000", 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0230000000000004e-06 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542366+0000", 2024-09-15T01:45:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013528014999999999 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542427+0000", 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0846e-05 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 76/53 ECSubWrite(tid=145, reqid=osd.4.0:145, at_version=76'94, trim_to=0'0, roll_forward_to=75'92))", 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.294489781999999, 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012260155, 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531266+0000", 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019e-05 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531281+0000", 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4214e-05 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531290+0000", 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9290000000000004e-06 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532672+0000", 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013827100000000001 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532682+0000", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7960000000000007e-06 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543048+0000", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010366175 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543499+0000", 2024-09-15T01:45:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045100000000000001 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 76/66 ECSubWrite(tid=67, reqid=osd.2.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=76'21))", 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.290948385, 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013434306999999999, 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:45:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534790+0000", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8699999999999998e-06 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534798+0000", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534804+0000", 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7450000000000001e-06 2024-09-15T01:45:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535012+0000", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020819 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535021+0000", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.473e-06 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548169+0000", 2024-09-15T01:45:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013148529000000001 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548215+0000", 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5666e-05 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:807 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.222184586, 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012342736999999999, 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 807 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603547+0000", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9380000000000001e-06 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603562+0000", 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4431e-05 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603604+0000", 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603622+0000", 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7706e-05 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603719+0000", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7546000000000004e-05 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603728+0000", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608524+0000", 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047951790000000001 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608636+0000", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011230600000000001 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.609358+0000", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072179799999999995 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613504+0000", 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041461270000000003 2024-09-15T01:45:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615819+0000", 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023155229999999999 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615887+0000", 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7708999999999998e-05 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:808 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [rollback 30] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.2220537760000001, 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012866819999999999, 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 808 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603677+0000", 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603684+0000", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3220000000000004e-06 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603705+0000", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1056999999999998e-05 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603716+0000", 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1073999999999999e-05 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603799+0000", 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3174e-05 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603803+0000", 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.422e-06 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610165+0000", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006361841 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610363+0000", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019824 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.611276+0000", 2024-09-15T01:45:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091360300000000003 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613506+0000", 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022295459999999998 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616474+0000", 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029682599999999999 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616542+0000", 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7694999999999995e-05 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:809 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.2219438059999996, 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015369404999999999, 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 809 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603787+0000", 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603793+0000", 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.951e-06 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603822+0000", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8274000000000001e-05 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603833+0000", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1243000000000001e-05 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603863+0000", 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0425000000000001e-05 2024-09-15T01:45:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603867+0000", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2030000000000001e-06 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612025+0000", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081579340000000004 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612192+0000", 2024-09-15T01:45:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016787099999999999 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613063+0000", 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087049600000000001 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615910+0000", 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028468719999999999 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619073+0000", 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031635119999999998 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619154+0000", 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1206999999999997e-05 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=159, reqid=client.4632.0:823, at_version=83'106, trim_to=0'0, roll_forward_to=80'103))", 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.2744069710000003, 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011255777999999999, 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551323+0000", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2160000000000001e-06 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551804+0000", 2024-09-15T01:45:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048123799999999999 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551819+0000", 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551827+0000", 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6499999999999996e-06 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551880+0000", 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3767999999999997e-05 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551943+0000", 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2779000000000003e-05 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562559+0000", 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010615820999999999 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562578+0000", 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=169, reqid=client.4632.0:845, at_version=83'112, trim_to=0'0, roll_forward_to=83'110))", 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1916235430000004, 2024-09-15T01:45:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012514281, 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634107+0000", 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.31e-06 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634298+0000", 2024-09-15T01:45:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019108600000000001 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634338+0000", 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300999999999999e-05 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634352+0000", 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3866000000000001e-05 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634422+0000", 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0359999999999997e-05 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634432+0000", 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5310000000000007e-06 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646573+0000", 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012141322 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646620+0000", 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6505000000000003e-05 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:06.943 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:06.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:45:06.945 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:06.943+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:06.970 INFO:tasks.rados.rados.0.smithi033.stdout:358: finishing copy_from to smithi03327189-38 2024-09-15T01:45:06.970 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 38 v 49 (ObjNum 107 snap 13 seq_num 107) dirty exists 2024-09-15T01:45:06.970 INFO:tasks.rados.rados.0.smithi033.stdout:356: finishing copy_from racing read to smithi03327189-28 2024-09-15T01:45:06.970 INFO:tasks.rados.rados.0.smithi033.stdout:356: finishing copy_from to smithi03327189-28 2024-09-15T01:45:06.971 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 28 v 64 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T01:45:07.045 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:07.042+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:07.055 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:07.055 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:07.055 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:07.055 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:07.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:45:07.146 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:07.142+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:07.169 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:07.169 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:07.169 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:07.169 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:07.169 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:07.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:45:07.247 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:07.242+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.283 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.759488249, 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:45:07.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:45:07.286 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.287 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:45:07.288 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:07.289 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.759429902999997, 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.291 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:45:07.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.293 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.294 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.759351146, 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:45:07.297 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.298 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:45:07.299 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 20.761265893000001, 2024-09-15T01:45:07.300 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.302 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.303 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:45:07.304 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.305 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.306 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:773 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 30] snapc 2d=[2a] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.7591843659999995, 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026472887000000001, 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 773 2024-09-15T01:45:07.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.308 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518960+0000", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1769999999999998e-06 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518990+0000", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519067+0000", 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7348000000000006e-05 2024-09-15T01:45:07.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519085+0000", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8050000000000002e-05 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519275+0000", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018974899999999999 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519561+0000", 2024-09-15T01:45:07.310 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028586399999999999 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.539158+0000", 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019596921999999999 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544838+0000", 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056797690000000003 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.311 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545308+0000", 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00047005099999999998 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545429+0000", 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121575 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:774 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:07.312 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.7590222000000004, 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025029231999999998, 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 774 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.313 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519121+0000", 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2330000000000001e-06 2024-09-15T01:45:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519129+0000", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7700000000000007e-06 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519153+0000", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3017e-05 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519164+0000", 2024-09-15T01:45:07.315 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1953000000000001e-05 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519853+0000", 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068810500000000005 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519993+0000", 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037699999999999 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.316 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.543813+0000", 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023819931999999999 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544004+0000", 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019107799999999999 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544148+0000", 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014376699999999999 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.317 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:821 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~753664] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.745535973, 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024406196000000002, 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 821 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.318 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532607+0000", 2024-09-15T01:45:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1720000000000001e-06 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533760+0000", 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001152761 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533784+0000", 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4366e-05 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.320 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533795+0000", 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1077000000000001e-05 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533897+0000", 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000102403 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533911+0000", 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3076e-05 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.536711+0000", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028006160000000001 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537117+0000", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040575600000000001 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.539897+0000", 2024-09-15T01:45:07.322 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027803419999999999 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552030+0000", 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012132830000000001 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.556945+0000", 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049147720000000004 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557011+0000", 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6025000000000006e-05 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:822 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~786432] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:07.324 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.7409531000000005, 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022305604, 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 822 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537190+0000", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.373e-06 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538247+0000", 2024-09-15T01:45:07.326 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001056981 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538272+0000", 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4587e-05 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538284+0000", 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1972e-05 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.327 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543940+0000", 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056560489999999998 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.544202+0000", 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026208399999999999 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.547032+0000", 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028303849999999999 2024-09-15T01:45:07.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552099+0000", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005067289 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559392+0000", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072926639999999999 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559493+0000", 2024-09-15T01:45:07.329 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010122 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:823 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~655360] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.7359520450000003, 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020918839000000002, 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.330 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 823 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542192+0000", 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9799999999999998e-06 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543766+0000", 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0015748540000000001 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.332 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543797+0000", 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0658999999999999e-05 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543841+0000", 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4029000000000001e-05 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548696+0000", 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004854985 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548823+0000", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012638100000000001 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.550883+0000", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002060596 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557375+0000", 2024-09-15T01:45:07.334 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0064918859999999997 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563026+0000", 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056509919999999996 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.335 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563107+0000", 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1477000000000006e-05 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:824 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~114688] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.7342800130000002, 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021066457, 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.336 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 824 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.337 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543863+0000", 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0669999999999999e-06 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545718+0000", 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001855675 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.338 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545744+0000", 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5227000000000001e-05 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545753+0000", 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3239999999999992e-06 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.339 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552985+0000", 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072324340000000003 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.553163+0000", 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017793400000000001 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.554394+0000", 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001230411 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557432+0000", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030381689999999999 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564820+0000", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073879770000000004 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564927+0000", 2024-09-15T01:45:07.341 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010723899999999999 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:826 3.0s0 3:08ecf469:::smithi03327189-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.7323561759999997, 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020342466, 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.342 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 826 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:07.343 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7899999999999999e-07 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545787+0000", 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6980000000000001e-06 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.344 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545791+0000", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1069999999999998e-06 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545793+0000", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.278e-06 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557250+0000", 2024-09-15T01:45:07.345 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011456579 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557367+0000", 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0001172 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566116+0000", 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087490199999999997 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.346 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566127+0000", 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1205e-05 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:855 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 161] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.6427170789999996, 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.040891745, 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.347 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 855 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.348 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635425+0000", 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.189e-06 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635431+0000", 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635447+0000", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5540999999999999e-05 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635455+0000", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8159999999999994e-06 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635541+0000", 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.6370000000000001e-05 2024-09-15T01:45:07.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635650+0000", 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109366 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.663664+0000", 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028014127999999999 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.351 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.667487+0000", 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038226110000000001 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676257+0000", 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087697060000000004 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676315+0000", 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8604000000000003e-05 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.352 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:856 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 4.6423320859999997, 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029427498999999999, 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 856 2024-09-15T01:45:07.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:07.354 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635814+0000", 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2710000000000003e-06 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635824+0000", 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9879999999999999e-06 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.355 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635830+0000", 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0870000000000003e-06 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635890+0000", 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9703000000000001e-05 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635946+0000", 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6369000000000002e-05 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.356 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.664989+0000", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029042743999999999 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665110+0000", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121499 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665236+0000", 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594599999999999 2024-09-15T01:45:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:901 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 49] snapc 36=[36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.689081837, 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029140195000000001, 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.358 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 901 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589062+0000", 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7829999999999999e-06 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589075+0000", 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3696000000000001e-05 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.360 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589110+0000", 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4675999999999998e-05 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589124+0000", 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3977e-05 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589270+0000", 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014596999999999999 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.361 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589474+0000", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020429199999999999 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.606938+0000", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017463523000000002 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612856+0000", 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0059182130000000003 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618075+0000", 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052193589999999998 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618199+0000", 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012370600000000001 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:902 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 1.6889936059999999, 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023705106, 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 902 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589148+0000", 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.327e-06 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589157+0000", 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.8789999999999992e-06 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.366 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589169+0000", 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2130000000000001e-05 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589176+0000", 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6139999999999997e-06 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589740+0000", 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056363300000000002 2024-09-15T01:45:07.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589833+0000", 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3189000000000005e-05 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612431+0000", 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022597926000000001 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612658+0000", 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00022690899999999999 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612852+0000", 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194499 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:920 3.3s0 3:c5b7678e:::smithi03327189-28:head [copy-from ver 64] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.69006290800000003, 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035195668999999999, 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 920 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588081+0000", 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588093+0000", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2293000000000001e-05 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588129+0000", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5559999999999998e-05 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588143+0000", 2024-09-15T01:45:07.372 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4185999999999999e-05 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588287+0000", 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014458299999999999 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588563+0000", 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00027543100000000003 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.373 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.613913+0000", 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025350342000000001 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.616276+0000", 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002362496 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623181+0000", 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0069057119999999996 2024-09-15T01:45:07.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623273+0000", 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2150000000000004e-05 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:921 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.68996626699999997, 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027587708999999998, 2024-09-15T01:45:07.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 921 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588176+0000", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588182+0000", 2024-09-15T01:45:07.377 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2999999999999998e-06 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588210+0000", 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7872e-05 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588223+0000", 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3183999999999999e-05 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588905+0000", 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068206500000000004 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589055+0000", 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014992299999999999 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615540+0000", 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026484735999999998 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615634+0000", 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3851999999999995e-05 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615762+0000", 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012850700000000001 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:924 3.cs0 3:3cf1f308:::smithi03327189-35:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.68965532100000004, 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023930099999999999, 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 924 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588487+0000", 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.254e-06 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588494+0000", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588512+0000", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8343e-05 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588522+0000", 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.03e-05 2024-09-15T01:45:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.591976+0000", 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034535500000000001 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.592256+0000", 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000280115 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.612415+0000", 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020159355 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:925 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 49] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.68959652199999999, 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027142459000000001, 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 925 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588546+0000", 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.42e-06 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588553+0000", 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8800000000000008e-06 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588572+0000", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8966000000000001e-05 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588584+0000", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1199999999999999e-05 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588679+0000", 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5367999999999999e-05 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588816+0000", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000137198 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.607033+0000", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018216392000000001 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.611398+0000", 2024-09-15T01:45:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0043651899999999997 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615605+0000", 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042073969999999999 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615687+0000", 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1447999999999997e-05 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:926 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 0.68952349999999996, 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021973917999999999, 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 926 2024-09-15T01:45:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588618+0000", 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.203e-06 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588624+0000", 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7930000000000003e-06 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588642+0000", 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8114e-05 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588653+0000", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0730000000000001e-05 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589048+0000", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039461299999999998 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589152+0000", 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000104838 2024-09-15T01:45:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610379+0000", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021226630999999999 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610491+0000", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000111718 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610591+0000", 2024-09-15T01:45:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100278 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:07.396 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:07.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:45:07.398 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:07.395+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:07.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:45:07.499 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:07.494+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:07.505 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:07.505 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:07.505 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:07.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:45:07.590 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:07.592 INFO:tasks.rados.rados.0.smithi033.stdout:354: done (5 left) 2024-09-15T01:45:07.592 INFO:tasks.rados.rados.0.smithi033.stdout:355: done (4 left) 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:356: done (3 left) 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:357: done (2 left) 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:358: done (1 left) 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:359: done (0 left) 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 13 current snap is 30 2024-09-15T01:45:07.593 INFO:tasks.rados.rados.0.smithi033.stdout:360: seq_num 148 ranges {1769472=81920} 2024-09-15T01:45:07.599 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:07.590+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is set 2024-09-15T01:45:07.605 INFO:tasks.rados.rados.0.smithi033.stdout:360: writing smithi03327189-13 from 1769472 to 1851392 tid 1 2024-09-15T01:45:07.605 INFO:tasks.rados.rados.0.smithi033.stdout:361: snap_remove snap 27 2024-09-15T01:45:07.623 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:07.623 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:07.623 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:07.624 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:07.624 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:07.624 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:07.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:45:07.641 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:07.660 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:07.681 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:07.700 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:07.694+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:07.703 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:07.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:07.730 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:07.731 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:07.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:45:07.746 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:07.768 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:07.792 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:07.801 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:07.794+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:07.815 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:07.838 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:07.840 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:07.840 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:07.841 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:07.842 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:07.842 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:07.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:45:07.901 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:07.899+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:07.955 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:07.963 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:07.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:45:08.003 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:07.994+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:08.104 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:08.102+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:08.114 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.114 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:45:08.205 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:08.202+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:08.225 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.225 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:45:08.306 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:08.304+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:08.337 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.337 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.346 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:45:08.407 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:08.403+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:08.450 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.450 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:45:08.508 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:08.502+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:08.566 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.566 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:45:08.610 INFO:tasks.rados.rados.0.smithi033.stdout:361: done (1 left) 2024-09-15T01:45:08.610 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 2 current snap is 30 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:360: finishing write tid 1 to smithi03327189-13 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:362: seq_num 149 ranges {0=98304} 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:362: writing smithi03327189-2 from 0 to 98304 tid 1 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:363: read oid 8 snap -1 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:363: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:364: read oid 23 snap -1 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:364: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:45:08.611 INFO:tasks.rados.rados.0.smithi033.stdout:365: snap_remove snap 25 2024-09-15T01:45:08.611 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:08.602+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:08.681 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.681 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.689 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:45:08.711 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:08.707+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:08.812 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:08.807+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:08.837 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.837 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:45:08.914 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:08.911+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:08.951 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:08.951 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:08.960 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:45:09.015 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:09.007+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:09.060 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.060 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.069 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:45:09.116 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:09.111+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:09.169 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.169 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.179 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:45:09.217 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:09.215+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:09.280 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.281 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.290 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:45:09.317 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:09.315+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:09.384 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.385 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.393 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:45:09.418 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:09.410+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:09.488 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.488 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.497 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:45:09.519 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:09.511+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:09.593 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.593 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.602 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:365: done (4 left) 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:366: read oid 7 snap 29 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:366: expect (ObjNum 118 snap 14 seq_num 118) 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:367: read oid 37 snap -1 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:367: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:45:09.616 INFO:tasks.rados.rados.0.smithi033.stdout:368: snap_remove snap 26 2024-09-15T01:45:09.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:45:09.619 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:09.610+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:09.705 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.705 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:45:09.720 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:09.715+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:09.806 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:45:09.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:45:09.807 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-15T01:45:09.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:09.809 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.809 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:45:09.821 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:09.815+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:09.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:45:09.921 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:09.915+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:09.954 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:09.954 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:09.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:45:09.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:10.011 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:10.022 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:10.019+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:10.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:10.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:10.059 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:10.060 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:45:10.070 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:10.101 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:10.114 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:10.123 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:10.115+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:10.133 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-15T01:45:09.611705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117604915,"num_objects":60,"num_object_clones":9,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":1,"num_read":467,"num_read_kb":193794,"num_write":890,"num_write_kb":188952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":39703091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1016,"ondisk_log_size":1016,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":10,"kb":943718400,"kb_used":3254116,"kb_used_data":238928,"kb_used_omap":16,"kb_used_meta":3015023,"kb_avail":940464284,"statfs":{"total":966367641600,"available":963035426816,"internally_reserved":0,"allocated":244662272,"data_stored":267908085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3087383920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15482880,"num_objects":6,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":54,"num_read_kb":66749,"num_write":134,"num_write_kb":33248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027820"},"pg_stats":[{"pgid":"3.f","version":"83'34","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T01:45:06.539860+0000","last_change":"2024-09-15T01:45:02.546296+0000","last_active":"2024-09-15T01:45:06.539860+0000","last_peered":"2024-09-15T01:45:06.539860+0000","last_clean":"2024-09-15T01:45:06.539860+0000","last_became_active":"2024-09-15T01:44:14.525529+0000","last_became_peered":"2024-09-15T01:44:14.525529+0000","last_unstale":"2024-09-15T01:45:06.539860+0000","last_undegraded":"2024-09-15T01:45:06.539860+0000","last_fullsized":"2024-09-15T01:45:06.539860+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154448,"stat_sum":{"num_bytes":4554752,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":31501,"num_write":25,"num_write_kb":12240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,1,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"30","length":"2"}]},{"pgid":"3.e","version":"84'53","reported_seq":257,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.576305+0000","last_change":"2024-09-15T01:45:02.570214+0000","last_active":"2024-09-15T01:45:04.576305+0000","last_peered":"2024-09-15T01:45:04.576305+0000","last_clean":"2024-09-15T01:45:04.576305+0000","last_became_active":"2024-09-15T01:44:14.525437+0000","last_became_peered":"2024-09-15T01:44:14.525437+0000","last_unstale":"2024-09-15T01:45:04.576305+0000","last_undegraded":"2024-09-15T01:45:04.576305+0000","last_fullsized":"2024-09-15T01:45:04.576305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.758952+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013283413000000001,"stat_sum":{"num_bytes":14336000,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":17480,"num_write":31,"num_write_kb":14800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.d","version":"85'36","reported_seq":303,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T01:45:06.526386+0000","last_change":"2024-09-15T01:45:02.545564+0000","last_active":"2024-09-15T01:45:06.526386+0000","last_peered":"2024-09-15T01:45:06.526386+0000","last_clean":"2024-09-15T01:45:06.526386+0000","last_became_active":"2024-09-15T01:44:33.552142+0000","last_became_peered":"2024-09-15T01:44:33.552142+0000","last_unstale":"2024-09-15T01:45:06.526386+0000","last_undegraded":"2024-09-15T01:45:06.526386+0000","last_fullsized":"2024-09-15T01:45:06.526386+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.473466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074899700000000005,"stat_sum":{"num_bytes":7782400,"num_objects":6,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":19,"num_read_kb":9956,"num_write":43,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3080192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,9,6,1,3,4,0],"acting":[7,10,9,6,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"30","length":"2"}]},{"pgid":"3.c","version":"76'62","reported_seq":297,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.592733+0000","last_change":"2024-09-15T01:45:02.541030+0000","last_active":"2024-09-15T01:45:04.592733+0000","last_peered":"2024-09-15T01:45:04.592733+0000","last_clean":"2024-09-15T01:45:04.592733+0000","last_became_active":"2024-09-15T01:44:33.554020+0000","last_became_peered":"2024-09-15T01:44:33.554020+0000","last_unstale":"2024-09-15T01:45:04.592733+0000","last_undegraded":"2024-09-15T01:45:04.592733+0000","last_fullsized":"2024-09-15T01:45:04.592733+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:15.891059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044460039999999999,"stat_sum":{"num_bytes":4177920,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":107,"num_write":65,"num_write_kb":12080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,10,9,2,0,7],"acting":[4,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.b","version":"83'49","reported_seq":272,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T01:45:05.587097+0000","last_change":"2024-09-15T01:45:02.549148+0000","last_active":"2024-09-15T01:45:05.587097+0000","last_peered":"2024-09-15T01:45:05.587097+0000","last_clean":"2024-09-15T01:45:05.587097+0000","last_became_active":"2024-09-15T01:44:14.534602+0000","last_became_peered":"2024-09-15T01:44:14.534602+0000","last_unstale":"2024-09-15T01:45:05.587097+0000","last_undegraded":"2024-09-15T01:45:05.587097+0000","last_fullsized":"2024-09-15T01:45:05.587097+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:17.646194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001942233,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":11745,"num_write":52,"num_write_kb":8208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.a","version":"84'22","reported_seq":194,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.690062+0000","last_change":"2024-09-15T01:45:02.561241+0000","last_active":"2024-09-15T01:45:04.690062+0000","last_peered":"2024-09-15T01:45:04.690062+0000","last_clean":"2024-09-15T01:45:04.690062+0000","last_became_active":"2024-09-15T01:44:46.539838+0000","last_became_peered":"2024-09-15T01:44:46.539838+0000","last_unstale":"2024-09-15T01:45:04.690062+0000","last_undegraded":"2024-09-15T01:45:04.690062+0000","last_fullsized":"2024-09-15T01:45:04.690062+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:06.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047142010000000003,"stat_sum":{"num_bytes":2899968,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":9063,"num_write":28,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,10,8,9,3,7],"acting":[2,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.9","version":"79'44","reported_seq":246,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583078+0000","last_change":"2024-09-15T01:45:02.547736+0000","last_active":"2024-09-15T01:45:04.583078+0000","last_peered":"2024-09-15T01:45:04.583078+0000","last_clean":"2024-09-15T01:45:04.583078+0000","last_became_active":"2024-09-15T01:44:33.554096+0000","last_became_peered":"2024-09-15T01:44:33.554096+0000","last_unstale":"2024-09-15T01:45:04.583078+0000","last_undegraded":"2024-09-15T01:45:04.583078+0000","last_fullsized":"2024-09-15T01:45:04.583078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:22.584117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092546999999999996,"stat_sum":{"num_bytes":4325376,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":18,"num_write_kb":5456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,0,1,4,10],"acting":[8,3,7,9,0,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.8","version":"83'29","reported_seq":271,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.582506+0000","last_change":"2024-09-15T01:45:02.546828+0000","last_active":"2024-09-15T01:45:04.582506+0000","last_peered":"2024-09-15T01:45:04.582506+0000","last_clean":"2024-09-15T01:45:04.582506+0000","last_became_active":"2024-09-15T01:44:33.555455+0000","last_became_peered":"2024-09-15T01:44:33.555455+0000","last_unstale":"2024-09-15T01:45:04.582506+0000","last_undegraded":"2024-09-15T01:45:04.582506+0000","last_fullsized":"2024-09-15T01:45:04.582506+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:12.770472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019903329999999999,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":4181,"num_write":16,"num_write_kb":4976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,9,10,7],"acting":[8,1,2,4,3,9,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"1.0","version":"21'92","reported_seq":222,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T01:45:06.525948+0000","last_change":"2024-09-15T01:45:01.074698+0000","last_active":"2024-09-15T01:45:06.525948+0000","last_peered":"2024-09-15T01:45:06.525948+0000","last_clean":"2024-09-15T01:45:06.525948+0000","last_became_active":"2024-09-15T01:44:14.520194+0000","last_became_peered":"2024-09-15T01:44:14.520194+0000","last_unstale":"2024-09-15T01:45:06.525948+0000","last_undegraded":"2024-09-15T01:45:06.525948+0000","last_fullsized":"2024-09-15T01:45:06.525948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:18.683034+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.576091+0000","last_change":"2024-09-15T01:45:03.802676+0000","last_active":"2024-09-15T01:45:04.576091+0000","last_peered":"2024-09-15T01:45:04.576091+0000","last_clean":"2024-09-15T01:45:04.576091+0000","last_became_active":"2024-09-15T01:44:30.530642+0000","last_became_peered":"2024-09-15T01:44:30.530642+0000","last_unstale":"2024-09-15T01:45:04.576091+0000","last_undegraded":"2024-09-15T01:45:04.576091+0000","last_fullsized":"2024-09-15T01:45:04.576091+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:08.522578+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"84'64","reported_seq":289,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.582863+0000","last_change":"2024-09-15T01:45:02.565032+0000","last_active":"2024-09-15T01:45:04.582863+0000","last_peered":"2024-09-15T01:45:04.582863+0000","last_clean":"2024-09-15T01:45:04.582863+0000","last_became_active":"2024-09-15T01:44:33.553924+0000","last_became_peered":"2024-09-15T01:44:33.553924+0000","last_unstale":"2024-09-15T01:45:04.582863+0000","last_undegraded":"2024-09-15T01:45:04.582863+0000","last_fullsized":"2024-09-15T01:45:04.582863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:10.985856+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.017210151999999999,"stat_sum":{"num_bytes":2228224,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":5431,"num_write":62,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,8,1,2,10],"acting":[3,9,6,4,8,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T01:45:06.525743+0000","last_change":"2024-09-15T01:43:57.903604+0000","last_active":"2024-09-15T01:45:06.525743+0000","last_peered":"2024-09-15T01:45:06.525743+0000","last_clean":"2024-09-15T01:45:06.525743+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:06.525743+0000","last_undegraded":"2024-09-15T01:45:06.525743+0000","last_fullsized":"2024-09-15T01:45:06.525743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:08.754773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'71","reported_seq":355,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.697093+0000","last_change":"2024-09-15T01:45:02.521741+0000","last_active":"2024-09-15T01:45:04.697093+0000","last_peered":"2024-09-15T01:45:04.697093+0000","last_clean":"2024-09-15T01:45:04.697093+0000","last_became_active":"2024-09-15T01:44:33.551949+0000","last_became_peered":"2024-09-15T01:44:33.551949+0000","last_unstale":"2024-09-15T01:45:04.697093+0000","last_undegraded":"2024-09-15T01:45:04.697093+0000","last_fullsized":"2024-09-15T01:45:04.697093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.694050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014067929999999999,"stat_sum":{"num_bytes":7946240,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":15661,"num_write":39,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8044544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,8,9,6,1],"acting":[0,4,10,3,8,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.575728+0000","last_change":"2024-09-15T01:43:57.408215+0000","last_active":"2024-09-15T01:45:04.575728+0000","last_peered":"2024-09-15T01:45:04.575728+0000","last_clean":"2024-09-15T01:45:04.575728+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:04.575728+0000","last_undegraded":"2024-09-15T01:45:04.575728+0000","last_fullsized":"2024-09-15T01:45:04.575728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"85'124","reported_seq":345,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.693068+0000","last_change":"2024-09-15T01:45:02.543745+0000","last_active":"2024-09-15T01:45:04.693068+0000","last_peered":"2024-09-15T01:45:04.693068+0000","last_clean":"2024-09-15T01:45:04.693068+0000","last_became_active":"2024-09-15T01:44:33.548478+0000","last_became_peered":"2024-09-15T01:44:33.548478+0000","last_unstale":"2024-09-15T01:45:04.693068+0000","last_undegraded":"2024-09-15T01:45:04.693068+0000","last_fullsized":"2024-09-15T01:45:04.693068+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.310629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074274359999999999,"stat_sum":{"num_bytes":16252928,"num_objects":8,"num_object_clones":2,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":44,"num_read_kb":10579,"num_write":127,"num_write_kb":26848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,6,1,7,8,0,3],"acting":[4,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.2","version":"23'2","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583092+0000","last_change":"2024-09-15T01:44:33.542100+0000","last_active":"2024-09-15T01:45:04.583092+0000","last_peered":"2024-09-15T01:45:04.583092+0000","last_clean":"2024-09-15T01:45:04.583092+0000","last_became_active":"2024-09-15T01:44:33.541651+0000","last_became_peered":"2024-09-15T01:44:33.541651+0000","last_unstale":"2024-09-15T01:45:04.583092+0000","last_undegraded":"2024-09-15T01:45:04.583092+0000","last_fullsized":"2024-09-15T01:45:04.583092+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:18.093385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"86'27","reported_seq":287,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T01:45:05.612776+0000","last_change":"2024-09-15T01:45:02.546496+0000","last_active":"2024-09-15T01:45:05.612776+0000","last_peered":"2024-09-15T01:45:05.612776+0000","last_clean":"2024-09-15T01:45:05.612776+0000","last_became_active":"2024-09-15T01:44:33.554140+0000","last_became_peered":"2024-09-15T01:44:33.554140+0000","last_unstale":"2024-09-15T01:45:05.612776+0000","last_undegraded":"2024-09-15T01:45:05.612776+0000","last_fullsized":"2024-09-15T01:45:05.612776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.139848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094215150000000001,"stat_sum":{"num_bytes":4390912,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1348,"num_write":15,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2277376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,8,9,10],"acting":[4,3,0,6,2,8,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"3.5","version":"83'168","reported_seq":443,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.593028+0000","last_change":"2024-09-15T01:45:02.566468+0000","last_active":"2024-09-15T01:45:04.593028+0000","last_peered":"2024-09-15T01:45:04.593028+0000","last_clean":"2024-09-15T01:45:04.593028+0000","last_became_active":"2024-09-15T01:44:33.551490+0000","last_became_peered":"2024-09-15T01:44:33.551490+0000","last_unstale":"2024-09-15T01:45:04.593028+0000","last_undegraded":"2024-09-15T01:45:04.593028+0000","last_fullsized":"2024-09-15T01:45:04.593028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:30.747962+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018127758000000001,"stat_sum":{"num_bytes":22593536,"num_objects":8,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":64,"num_read_kb":38046,"num_write":121,"num_write_kb":39648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5160960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,6,7,1,8,10],"acting":[9,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583886+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:45:04.583886+0000","last_peered":"2024-09-15T01:45:04.583886+0000","last_clean":"2024-09-15T01:45:04.583886+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:04.583886+0000","last_undegraded":"2024-09-15T01:45:04.583886+0000","last_fullsized":"2024-09-15T01:45:04.583886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"82'65","reported_seq":352,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583754+0000","last_change":"2024-09-15T01:45:02.547519+0000","last_active":"2024-09-15T01:45:04.583754+0000","last_peered":"2024-09-15T01:45:04.583754+0000","last_clean":"2024-09-15T01:45:04.583754+0000","last_became_active":"2024-09-15T01:44:33.548174+0000","last_became_peered":"2024-09-15T01:44:33.548174+0000","last_unstale":"2024-09-15T01:45:04.583754+0000","last_undegraded":"2024-09-15T01:45:04.583754+0000","last_fullsized":"2024-09-15T01:45:04.583754+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:11.395908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016100019999999999,"stat_sum":{"num_bytes":11862016,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":27622,"num_write":63,"num_write_kb":11856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9895936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,3,10,2,6,0],"acting":[1,8,9,3,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.592650+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:45:04.592650+0000","last_peered":"2024-09-15T01:45:04.592650+0000","last_clean":"2024-09-15T01:45:04.592650+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:45:04.592650+0000","last_undegraded":"2024-09-15T01:45:04.592650+0000","last_fullsized":"2024-09-15T01:45:04.592650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"72'34","reported_seq":256,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583379+0000","last_change":"2024-09-15T01:45:02.547537+0000","last_active":"2024-09-15T01:45:04.583379+0000","last_peered":"2024-09-15T01:45:04.583379+0000","last_clean":"2024-09-15T01:45:04.583379+0000","last_became_active":"2024-09-15T01:44:14.534165+0000","last_became_peered":"2024-09-15T01:44:14.534165+0000","last_unstale":"2024-09-15T01:45:04.583379+0000","last_undegraded":"2024-09-15T01:45:04.583379+0000","last_fullsized":"2024-09-15T01:45:04.583379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.075163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014328380000000001,"stat_sum":{"num_bytes":2048000,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1255,"num_write":36,"num_write_kb":2400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.583901+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:45:04.583901+0000","last_peered":"2024-09-15T01:45:04.583901+0000","last_clean":"2024-09-15T01:45:04.583901+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:04.583901+0000","last_undegraded":"2024-09-15T01:45:04.583901+0000","last_fullsized":"2024-09-15T01:45:04.583901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"64'38","reported_seq":271,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.693714+0000","last_change":"2024-09-15T01:45:02.522250+0000","last_active":"2024-09-15T01:45:04.693714+0000","last_peered":"2024-09-15T01:45:04.693714+0000","last_clean":"2024-09-15T01:45:04.693714+0000","last_became_active":"2024-09-15T01:44:33.551249+0000","last_became_peered":"2024-09-15T01:44:33.551249+0000","last_unstale":"2024-09-15T01:45:04.693714+0000","last_undegraded":"2024-09-15T01:45:04.693714+0000","last_fullsized":"2024-09-15T01:45:04.693714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:09.245186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012694640000000001,"stat_sum":{"num_bytes":4947968,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":9753,"num_write":24,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,6],"acting":[0,1,4,7,8,10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"1"},{"start":"30","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T01:45:04.594135+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:45:04.594135+0000","last_peered":"2024-09-15T01:45:04.594135+0000","last_clean":"2024-09-15T01:45:04.594135+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:04.594135+0000","last_undegraded":"2024-09-15T01:45:04.594135+0000","last_fullsized":"2024-09-15T01:45:04.594135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":117014528,"num_objects":56,"num_object_clones":9,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":1,"num_read":391,"num_read_kb":193730,"num_write":765,"num_write_kb":186800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":39174144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":248655872,"data_stored":276615168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":920,"ondisk_log_size":920,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411347,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322640,"kb_used_data":21120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94049200,"statfs":{"total":96636764160,"available":96306380800,"internally_reserved":0,"allocated":21626880,"data_stored":24459263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.1020000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.70899999999999996}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2809999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.353},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.3779999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.252},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.137}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.468},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.4299999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.323},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.405}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444049,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444050,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325624,"kb_used_data":24104,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046216,"statfs":{"total":96636764160,"available":96303325184,"internally_reserved":0,"allocated":24682496,"data_stored":26330023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.071}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.627},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.022},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.157}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1279999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.169},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0509999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.97499999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.3109999999999999}]}]},{"osd":9,"up_from":17,"seq":73014444049,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331084,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94040756,"statfs":{"total":96636764160,"available":96297734144,"internally_reserved":0,"allocated":30273536,"data_stored":33096615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1870000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.296},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.256},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4319999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.3740000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.45},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.871}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.4990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.8979999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444049,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327124,"kb_used_data":25608,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94044716,"statfs":{"total":96636764160,"available":96301789184,"internally_reserved":0,"allocated":26222592,"data_stored":28447358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.83999999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.81299999999999994}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.4139999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.034},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.262}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.29},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.329},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.92800000000000005}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.153},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0920000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.1890000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.7909999999999999}]}]},{"osd":6,"up_from":17,"seq":73014444050,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":324596,"kb_used_data":23076,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94047244,"statfs":{"total":96636764160,"available":96304377856,"internally_reserved":0,"allocated":23629824,"data_stored":26445274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.202},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.01}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.5609999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.2769999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.056},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.359}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.395},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.696}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.415},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.52}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.843},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.667}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.4910000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.887},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.911}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.722},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.7509999999999999}]}]},{"osd":5,"up_from":0,"seq":73014444050,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":17,"seq":73014444049,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322672,"kb_used_data":21152,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94049168,"statfs":{"total":96636764160,"available":96306348032,"internally_reserved":0,"allocated":21659648,"data_stored":24482727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.004},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3939999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.4750000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.105},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.79800000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.532},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.167}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.21},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.274},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.9890000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":2.0430000000000001},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.675}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":2.093}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444049,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328224,"kb_used_data":26708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043616,"statfs":{"total":96636764160,"available":96300662784,"internally_reserved":0,"allocated":27348992,"data_stored":30175870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.90600000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.016}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.167}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.99199999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.494},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.5840000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.5549999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.246},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.518}]}]},{"osd":2,"up_from":17,"seq":73014444050,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319520,"kb_used_data":18000,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052320,"statfs":{"total":96636764160,"available":96309575680,"internally_reserved":0,"allocated":18432000,"data_stored":18457511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.47199999999999998},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.72399999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86599999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.83199999999999996}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":0.79000000000000004}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":0.91700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444049,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328996,"kb_used_data":27476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94042844,"statfs":{"total":96636764160,"available":96299872256,"internally_reserved":0,"allocated":28135424,"data_stored":30954426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.80100000000000005}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.006}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.103},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.97099999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.45},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.0580000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.079},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.647}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":1.298},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.3200000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.222},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.599}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3939999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444049,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323636,"kb_used_data":22120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94048204,"statfs":{"total":96636764160,"available":96305360896,"internally_reserved":0,"allocated":22650880,"data_stored":25059018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.872}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.0980000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.282}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.454},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.054}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.421},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.5840000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.161},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.637}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.6839999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.55}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22183936,"data_stored":24981504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27111424,"data_stored":30330880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17412096,"data_stored":17833984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26877952,"data_stored":30097408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20639744,"data_stored":23859200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8859648,"data_stored":9601024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22073344,"data_stored":25292800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20049920,"data_stored":23289856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":25751552,"data_stored":28368896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":29253632,"data_stored":32473088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":23662592,"data_stored":25706496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:10.136 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-15T01:45:10.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-15T01:45:10.137 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:10.171 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.658236707, 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:45:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.655805309000002, 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:45:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.655566493999999, 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:45:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.655535110999999, 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:45:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.676327680999997, 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:45:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.668413278999999, 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:45:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:45:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.668384015999997, 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:45:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:45:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:45:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.668808697000003, 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:45:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.666570235999998, 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:45:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:45:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.666000566999998, 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:45:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:45:10.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.665941599, 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:45:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:45:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:10.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.678043391000003, 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:10.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:45:10.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:392 3.6s0 3:6579c50e:::smithi03327189-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 43.677949613000003, 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012121346, 2024-09-15T01:45:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 392 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506735+0000", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506736+0000", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2899999999999999e-06 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506740+0000", 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0419999999999996e-06 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506750+0000", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0385e-05 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506757+0000", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3969999999999999e-06 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507041+0000", 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000284023 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.507051+0000", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0349999999999999e-05 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511467+0000", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044164230000000001 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511579+0000", 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111524 2024-09-15T01:45:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518448+0000", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068694389999999998 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518649+0000", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020049299999999999 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.518856+0000", 2024-09-15T01:45:10.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020698 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=117, reqid=osd.4.0:117, at_version=68'74, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.670214221999998, 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011721416, 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514470+0000", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514471+0000", 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514541+0000", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0955000000000001e-05 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514567+0000", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.597e-05 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514584+0000", 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6463000000000001e-05 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514727+0000", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014272899999999999 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514754+0000", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7290000000000001e-05 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526159+0000", 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011404621 2024-09-15T01:45:10.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.526191+0000", 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2840999999999999e-05 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.cs6 68/53 ECSubWrite(tid=118, reqid=osd.4.0:118, at_version=68'50, trim_to=0'0, roll_forward_to=67'48))", 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.669806973, 2024-09-15T01:45:10.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012184878999999999, 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514877+0000", 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514879+0000", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4729999999999999e-06 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514938+0000", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9142999999999999e-05 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.514963+0000", 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5165999999999999e-05 2024-09-15T01:45:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515032+0000", 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8602999999999994e-05 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515160+0000", 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012877200000000001 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.515188+0000", 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7313000000000001e-05 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527007+0000", 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011819079999999999 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.527062+0000", 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5328999999999998e-05 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s6 68/53 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=68'76, trim_to=0'0, roll_forward_to=67'72))", 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.667752589999999, 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014580965, 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516932+0000", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516933+0000", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6249999999999999e-06 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.516977+0000", 2024-09-15T01:45:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3551999999999999e-05 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517002+0000", 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4762000000000001e-05 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517016+0000", 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.401e-05 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517789+0000", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00077353199999999995 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.517813+0000", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3748e-05 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531475+0000", 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013662636000000001 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:47.531513+0000", 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7100000000000001e-05 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:753 3.1s0 3:86165cc9:::smithi03327189-11:head [copy-from ver 13] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.583332609999999, 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022114891000000001, 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 753 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601354+0000", 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6000000000000001e-06 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601366+0000", 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601396+0000", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9901999999999999e-05 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601406+0000", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601519+0000", 2024-09-15T01:45:10.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112546 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601677+0000", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015800100000000001 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.614051+0000", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012374232000000001 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617994+0000", 2024-09-15T01:45:10.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039432089999999996 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623415+0000", 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054204240000000001 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623466+0000", 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1972000000000003e-05 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:754 3.1s0 3:86165cc9:::smithi03327189-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.583252963, 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015926781000000001, 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 754 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601433+0000", 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4270000000000001e-06 2024-09-15T01:45:10.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601437+0000", 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2899999999999996e-06 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601452+0000", 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4939999999999999e-05 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601461+0000", 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601979+0000", 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051790899999999999 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.602057+0000", 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8237000000000003e-05 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617204+0000", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015147361999999999 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617280+0000", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6318999999999999e-05 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617358+0000", 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7570000000000004e-05 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.290 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:915 3.6s0 3:6e27a4f5:::smithi03327189-22:head [copy-from ver 13] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5968646089999998, 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022788083000000001, 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 915 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587822+0000", 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6639999999999998e-06 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587838+0000", 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5894000000000001e-05 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587878+0000", 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9827999999999999e-05 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587893+0000", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4538999999999999e-05 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588032+0000", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000139982 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588210+0000", 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017786399999999999 2024-09-15T01:45:10.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.602185+0000", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013974268999999999 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.605186+0000", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030014400000000002 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.610512+0000", 2024-09-15T01:45:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053261050000000002 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.610608+0000", 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5497999999999999e-05 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:916 3.6s0 3:6e27a4f5:::smithi03327189-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5967260969999999, 2024-09-15T01:45:10.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017108080000000001, 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 916 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587959+0000", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1400000000000001e-06 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587965+0000", 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2719999999999997e-06 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587979+0000", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4554e-05 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587989+0000", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0196e-05 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588496+0000", 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050668000000000002 2024-09-15T01:45:10.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7958999999999999e-05 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.604843+0000", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016299200999999999 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.604955+0000", 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112382 2024-09-15T01:45:10.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.605066+0000", 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110696 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.301 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:45:10.303 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:10.303 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:10.304 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:10.304 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:10.305 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:10.299+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:10.403 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:10.395+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:10.446 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.447 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:10.447 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:10.447 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:45:10.506 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:10.503+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:10.562 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.562 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:10.562 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:10.562 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:10.562 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:45:10.606 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:10.602+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:10.634 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:45:10.635 INFO:tasks.rados.rados.0.smithi033.stdout:368: done (6 left) 2024-09-15T01:45:10.635 INFO:tasks.rados.rados.0.smithi033.stdout:369: delete oid 49 current snap is 30 2024-09-15T01:45:10.635 INFO:tasks.rados.rados.0.smithi033.stdout:360: finishing write tid 2 to smithi03327189-13 2024-09-15T01:45:10.635 INFO:tasks.rados.rados.0.smithi033.stdout:360: finishing write tid 3 to smithi03327189-13 2024-09-15T01:45:10.636 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 13 v 81 (ObjNum 148 snap 30 seq_num 148) dirty exists 2024-09-15T01:45:10.636 INFO:tasks.rados.rados.0.smithi033.stdout:360: left oid 13 (ObjNum 148 snap 30 seq_num 148) 2024-09-15T01:45:10.636 INFO:tasks.rados.rados.0.smithi033.stdout:364: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-15T01:45:10.641 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 3 pg_num to 26 2024-09-15T01:45:10.652 INFO:tasks.rados.rados.0.smithi033.stdout:360: done (6 left) 2024-09-15T01:45:10.652 INFO:tasks.rados.rados.0.smithi033.stdout:364: done (5 left) 2024-09-15T01:45:10.652 INFO:tasks.rados.rados.0.smithi033.stdout:369: done (4 left) 2024-09-15T01:45:10.652 INFO:tasks.rados.rados.0.smithi033.stdout:370: delete oid 18 current snap is 30 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:362: finishing write tid 1 to smithi03327189-2 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:362: finishing write tid 2 to smithi03327189-2 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:362: finishing write tid 3 to smithi03327189-2 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 2 v 88 (ObjNum 149 snap 30 seq_num 149) dirty exists 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:362: left oid 2 (ObjNum 149 snap 30 seq_num 149) 2024-09-15T01:45:10.653 INFO:tasks.rados.rados.0.smithi033.stdout:363: expect (ObjNum 106 snap 13 seq_num 106) 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:229 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.167633379000002, 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014782722, 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 229 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521025+0000", 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521026+0000", 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0800000000000004e-07 2024-09-15T01:45:10.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521029+0000", 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3299999999999999e-06 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521045+0000", 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6212999999999999e-05 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521050+0000", 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0300000000000001e-06 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526537+0000", 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054869560000000003 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.526747+0000", 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020936500000000001 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.535808+0000", 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00906122 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 58.167602434999999, 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:10.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:10.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.170722613999999, 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:45:10.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.170653082000001, 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:45:10.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.170630545999998, 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:45:10.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.170563354000002, 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:45:10.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:45:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:45:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:10.727 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.170493829000002, 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:45:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:10.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.170251751000002, 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:10.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:45:10.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:45:10.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.170198317000001, 2024-09-15T01:45:10.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:45:10.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:45:10.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.170126138000001, 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:45:10.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:45:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.164275458999999, 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:45:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:45:10.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.159358637, 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:45:10.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:45:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 73/53 ECSubWrite(tid=132, reqid=osd.4.0:132, at_version=73'84, trim_to=0'0, roll_forward_to=72'82))", 2024-09-15T01:45:10.761 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.171772667999999, 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015282341, 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516946+0000", 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9145999999999999e-05 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516976+0000", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0236e-05 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516993+0000", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517168+0000", 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017460800000000001 2024-09-15T01:45:10.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517197+0000", 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8813e-05 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532115+0000", 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014917629 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532168+0000", 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3596e-05 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:740 3.4s0 3:20577364:::smithi03327189-24:head [copy-from ver 0] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.169089366000001, 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016305848000000001, 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 740 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:10.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519573+0000", 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-06 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519590+0000", 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519639+0000", 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9432999999999999e-05 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519658+0000", 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9083e-05 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521437+0000", 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017789539999999999 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521452+0000", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4952e-05 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530454+0000", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.009001963 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530578+0000", 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012395599999999999 2024-09-15T01:45:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535875+0000", 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052969169999999999 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:741 3.4s0 3:20577364:::smithi03327189-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.168912078000002, 2024-09-15T01:45:10.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016411578999999999, 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 741 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519748+0000", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.04e-06 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519758+0000", 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5929999999999995e-06 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519795+0000", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6860000000000003e-05 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519810+0000", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5082999999999999e-05 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521558+0000", 2024-09-15T01:45:10.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017476 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521568+0000", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0344e-05 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530788+0000", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092198539999999995 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530875+0000", 2024-09-15T01:45:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7698e-05 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535967+0000", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918409999999997 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536066+0000", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8281000000000003e-05 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536158+0000", 2024-09-15T01:45:10.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2385000000000004e-05 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=183, reqid=client.4632.0:738, at_version=73'21, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.165118634999999, 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018357772000000001, 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523541+0000", 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.24e-06 2024-09-15T01:45:10.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523599+0000", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7500999999999997e-05 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523628+0000", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9193999999999999e-05 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523640+0000", 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2276000000000001e-05 2024-09-15T01:45:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531608+0000", 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079682079999999992 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531619+0000", 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0293e-05 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541866+0000", 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010247114999999999 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541898+0000", 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1945e-05 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 73/34 ECSubWrite(tid=58, reqid=osd.2.0:58, at_version=73'43, trim_to=0'0, roll_forward_to=72'41))", 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.162379311999999, 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015869550999999999, 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:10.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526280+0000", 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999996e-07 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526306+0000", 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6282e-05 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526322+0000", 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5977999999999998e-05 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526330+0000", 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.191e-06 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526451+0000", 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120526 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526459+0000", 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542088+0000", 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015629295000000001 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542149+0000", 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0247000000000001e-05 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:811 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~737280] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1602757879999999, 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022983506000000001, 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 811 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528386+0000", 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.292e-06 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529048+0000", 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066185599999999995 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529087+0000", 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9653999999999997e-05 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529102+0000", 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4931e-05 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529205+0000", 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103047 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529215+0000", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7159999999999998e-06 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533610+0000", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043948049999999999 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533964+0000", 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000354235 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.536912+0000", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029481400000000001 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551101+0000", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014188383000000001 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551169+0000", 2024-09-15T01:45:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8662000000000001e-05 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551366+0000", 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019678499999999999 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:812 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~770048] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.1549927879999995, 2024-09-15T01:45:10.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022084530000000002, 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 812 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533668+0000", 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5969999999999999e-06 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534580+0000", 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091206999999999996 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534606+0000", 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5573999999999999e-05 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534627+0000", 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1571000000000001e-05 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541493+0000", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068657910000000004 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541606+0000", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011307 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.543077+0000", 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014709670000000001 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.553529+0000", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010451313 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555682+0000", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021530949999999998 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555750+0000", 2024-09-15T01:45:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8481999999999999e-05 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:910 3.8s1 3:1086ba31:::smithi03327189-43:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.523350+0000", 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.1653088409999999, 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015039274, 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 910 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523350+0000", 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523350+0000", 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523350+0000", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8100000000000003e-07 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523354+0000", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6330000000000002e-06 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523365+0000", 2024-09-15T01:45:10.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0903999999999999e-05 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523371+0000", 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0739999999999998e-06 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523523+0000", 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015264900000000001 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.523530+0000", 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7220000000000002e-06 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.530219+0000", 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006688851 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.530385+0000", 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016635099999999999 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:10.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.538389+0000", 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080035090000000007 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:10.803 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:45:10.805 INFO:tasks.rados.rados.0.smithi033.stdout:367: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-15T01:45:10.805 INFO:tasks.rados.rados.0.smithi033.stdout:366: expect (ObjNum 118 snap 14 seq_num 118) 2024-09-15T01:45:10.806 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:10.799+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:10.863 INFO:tasks.rados.rados.0.smithi033.stdout:362: done (4 left) 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:363: done (3 left) 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:366: done (2 left) 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:367: done (1 left) 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:370: done (0 left) 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 12 current snap is 30 2024-09-15T01:45:10.864 INFO:tasks.rados.rados.0.smithi033.stdout:371: seq_num 152 ranges {2146304=147456} 2024-09-15T01:45:10.877 INFO:tasks.rados.rados.0.smithi033.stdout:371: writing smithi03327189-12 from 2146304 to 2293760 tid 1 2024-09-15T01:45:10.878 INFO:tasks.rados.rados.0.smithi033.stdout:372: read oid 48 snap -1 2024-09-15T01:45:10.878 INFO:tasks.rados.rados.0.smithi033.stdout:372: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:45:10.878 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 6 current snap is 30 2024-09-15T01:45:10.878 INFO:tasks.rados.rados.0.smithi033.stdout:373: seq_num 153 ranges {2408448=163840} 2024-09-15T01:45:10.896 INFO:tasks.rados.rados.0.smithi033.stdout:373: writing smithi03327189-6 from 2408448 to 2572288 tid 1 2024-09-15T01:45:10.897 INFO:tasks.rados.rados.0.smithi033.stdout:374: rollback oid 24 current snap is 30 2024-09-15T01:45:10.897 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 24 to 28 2024-09-15T01:45:10.897 INFO:tasks.rados.rados.0.smithi033.stdout:372: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-15T01:45:10.900 INFO:tasks.rados.rados.0.smithi033.stdout:371: finishing write tid 1 to smithi03327189-12 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:371: finishing write tid 2 to smithi03327189-12 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:371: finishing write tid 3 to smithi03327189-12 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 163 (ObjNum 152 snap 30 seq_num 152) dirty exists 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:371: left oid 12 (ObjNum 152 snap 30 seq_num 152) 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:371: done (3 left) 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:372: done (2 left) 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 10 current snap is 30 2024-09-15T01:45:10.901 INFO:tasks.rados.rados.0.smithi033.stdout:375: seq_num 154 ranges {1622016=180224} 2024-09-15T01:45:10.905 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:10.903+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:10.912 INFO:tasks.rados.rados.0.smithi033.stdout:375: writing smithi03327189-10 from 1622016 to 1802240 tid 1 2024-09-15T01:45:10.912 INFO:tasks.rados.rados.0.smithi033.stdout:376: rollback oid 22 current snap is 30 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 22 to 29 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:374: finishing rollback tid 0 to smithi03327189-24 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:373: finishing write tid 1 to smithi03327189-6 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:374: finishing rollback tid 1 to smithi03327189-24 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:374: finishing rollback tid 2 to smithi03327189-24 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 24 v 69 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:373: finishing write tid 2 to smithi03327189-6 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:374: done (3 left) 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:377: rollback oid 18 current snap is 30 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 18 to 29 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:373: finishing write tid 3 to smithi03327189-6 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 139 (ObjNum 153 snap 30 seq_num 153) dirty exists 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:373: left oid 6 (ObjNum 153 snap 30 seq_num 153) 2024-09-15T01:45:10.913 INFO:tasks.rados.rados.0.smithi033.stdout:373: done (3 left) 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:378: rollback oid 48 current snap is 30 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 48 to 28 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:379: read oid 9 snap 29 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:379: expect (ObjNum 100 snap 13 seq_num 100) 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:380: read oid 33 snap 29 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:380: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:45:10.914 INFO:tasks.rados.rados.0.smithi033.stdout:381: snap_create 2024-09-15T01:45:10.922 INFO:tasks.rados.rados.0.smithi033.stdout:378: finishing rollback tid 0 to smithi03327189-48 2024-09-15T01:45:10.922 INFO:tasks.rados.rados.0.smithi033.stdout:380: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-15T01:45:10.927 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:10.927 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:10.927 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:10.927 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:377: finishing rollback tid 1 to smithi03327189-18 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:377: finishing rollback tid 2 to smithi03327189-18 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 18 v 188 (ObjNum 94 snap 11 seq_num 94) dirty exists 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:375: finishing write tid 1 to smithi03327189-10 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:375: finishing write tid 2 to smithi03327189-10 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:378: finishing rollback tid 1 to smithi03327189-48 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:378: finishing rollback tid 2 to smithi03327189-48 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 48 v 69 (ObjNum 81 snap 0 seq_num 1733330416) dirty exists 2024-09-15T01:45:10.942 INFO:tasks.rados.rados.0.smithi033.stdout:376: finishing rollback tid 0 to smithi03327189-22 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:375: finishing write tid 3 to smithi03327189-10 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 10 v 83 (ObjNum 154 snap 30 seq_num 154) dirty exists 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:375: left oid 10 (ObjNum 154 snap 30 seq_num 154) 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:376: finishing rollback tid 1 to smithi03327189-22 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:376: finishing rollback tid 2 to smithi03327189-22 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 22 v 86 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-09-15T01:45:10.943 INFO:tasks.rados.rados.0.smithi033.stdout:379: expect (ObjNum 100 snap 13 seq_num 100) 2024-09-15T01:45:10.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:45:11.007 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:11.002+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:11.036 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:11.036 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:11.036 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:11.037 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:11.037 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:11.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:45:11.107 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:11.103+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.625954047, 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:45:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.497890423999998, 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:45:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:45:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.396926299, 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:45:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:45:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.393486725000002, 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:45:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:45:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.630543166999999, 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:45:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:45:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.630492246999999, 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:45:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:45:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.630427236999999, 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:45:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.615254628999999, 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:45:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.611210563, 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:45:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:817 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6076130319999997, 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012084411, 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 817 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540086+0000", 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1380000000000001e-06 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540094+0000", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5210000000000002e-06 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540112+0000", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7937000000000001e-05 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540121+0000", 2024-09-15T01:45:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0319999999999992e-06 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540573+0000", 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045140399999999998 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540597+0000", 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4651000000000002e-05 2024-09-15T01:45:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546537+0000", 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059398979999999999 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546648+0000", 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110967 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552168+0000", 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055198629999999999 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:818 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.607555756, 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012454458, 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 818 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540142+0000", 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540146+0000", 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0489999999999997e-06 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540242+0000", 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6168999999999994e-05 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540256+0000", 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4063000000000001e-05 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540737+0000", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048110699999999999 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540764+0000", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7509999999999999e-05 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550905+0000", 2024-09-15T01:45:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010140697000000001 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550984+0000", 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8906000000000007e-05 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552595+0000", 2024-09-15T01:45:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016113309999999999 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:819 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.6074217060000002, 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012574444000000001, 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 819 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540276+0000", 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-06 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540284+0000", 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.605e-06 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540297+0000", 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3526000000000001e-05 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540304+0000", 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2249999999999997e-06 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541504+0000", 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012007179999999999 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541512+0000", 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1099999999999997e-06 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551300+0000", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097880039999999995 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551387+0000", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7618e-05 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552849+0000", 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001462281 2024-09-15T01:45:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:832 3.as0 3:5045eccb:::smithi03327189-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5953496860000005, 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010647103, 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 832 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552348+0000", 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2079999999999999e-06 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552356+0000", 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9300000000000003e-06 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552373+0000", 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6677999999999999e-05 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552381+0000", 2024-09-15T01:45:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552513+0000", 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013210000000000001 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552519+0000", 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9020000000000004e-06 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557555+0000", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050360889999999997 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557647+0000", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2503000000000005e-05 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562994+0000", 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053469040000000004 2024-09-15T01:45:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:917 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5599314399999997, 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013596409, 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 917 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587768+0000", 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4480000000000001e-06 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587781+0000", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2958e-05 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587818+0000", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7128000000000001e-05 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587833+0000", 2024-09-15T01:45:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5404999999999999e-05 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587920+0000", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6885999999999994e-05 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588036+0000", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116384 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601362+0000", 2024-09-15T01:45:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013325200000000001 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:918 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.559831773, 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018684564000000001, 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 918 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587871+0000", 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1920000000000002e-06 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587878+0000", 2024-09-15T01:45:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.72e-06 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587893+0000", 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5141e-05 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587901+0000", 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588509+0000", 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00060832400000000004 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588593+0000", 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3774000000000001e-05 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606549+0000", 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017956577000000001 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:919 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5597853439999998, 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024313863000000002, 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 919 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587912+0000", 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.16e-07 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587917+0000", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.442e-06 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587925+0000", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2160000000000006e-06 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587928+0000", 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6179999999999999e-06 2024-09-15T01:45:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.592395+0000", 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044660990000000003 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.592821+0000", 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042640199999999999 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.612225+0000", 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019404270000000001 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 87/34 ECSubReadReply(tid=90, attrs_read=0))", 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5520370400000001, 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011032053, 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601572+0000", 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059121219999999997 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601581+0000", 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3690000000000007e-06 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601584+0000", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8299999999999998e-06 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601881+0000", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029641800000000001 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606692+0000", 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048108049999999996 2024-09-15T01:45:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 87/34 ECSubReadReply(tid=91, attrs_read=0))", 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5460983979999998, 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010736961, 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601599+0000", 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4099999999999997e-07 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606083+0000", 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044838739999999997 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606100+0000", 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6918999999999998e-05 2024-09-15T01:45:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606107+0000", 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6139999999999997e-06 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606753+0000", 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00064655699999999997 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.612335+0000", 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055820560000000002 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:949 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 38=[38,37] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.51676533499999999, 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013716572, 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 949 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630933+0000", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1050000000000002e-06 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630941+0000", 2024-09-15T01:45:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9500000000000001e-06 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630965+0000", 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4029e-05 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630977+0000", 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1163e-05 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.631731+0000", 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00075429399999999997 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.631986+0000", 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025550099999999999 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.632863+0000", 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087646399999999998 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.635373+0000", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002509945 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.644561+0000", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091884740000000003 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.644648+0000", 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6646999999999999e-05 2024-09-15T01:45:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 91/53 ECSubWrite(tid=159, reqid=osd.9.0:159, at_version=91'184, trim_to=0'0, roll_forward_to=90'180))", 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.51313821699999995, 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011302095, 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634559+0000", 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634587+0000", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7236000000000001e-05 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634598+0000", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1925e-05 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634605+0000", 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8859999999999998e-06 2024-09-15T01:45:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634651+0000", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5655000000000002e-05 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634661+0000", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0186999999999999e-05 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.645840+0000", 2024-09-15T01:45:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011179011000000001 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.645861+0000", 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0389000000000001e-05 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.259 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:11.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:45:11.261 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:11.259+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:11.362 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:11.359+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:11.368 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:11.368 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:11.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:45:11.462 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:11.455+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:11.477 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:11.477 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:11.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:45:11.521 INFO:tasks.rados.rados.0.smithi033.stdout:375: done (6 left) 2024-09-15T01:45:11.521 INFO:tasks.rados.rados.0.smithi033.stdout:376: done (5 left) 2024-09-15T01:45:11.521 INFO:tasks.rados.rados.0.smithi033.stdout:377: done (4 left) 2024-09-15T01:45:11.521 INFO:tasks.rados.rados.0.smithi033.stdout:378: done (3 left) 2024-09-15T01:45:11.521 INFO:tasks.rados.rados.0.smithi033.stdout:379: done (2 left) 2024-09-15T01:45:11.522 INFO:tasks.rados.rados.0.smithi033.stdout:380: done (1 left) 2024-09-15T01:45:11.522 INFO:tasks.rados.rados.0.smithi033.stdout:381: done (0 left) 2024-09-15T01:45:11.522 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 4 current snap is 31 2024-09-15T01:45:11.522 INFO:tasks.rados.rados.0.smithi033.stdout:382: seq_num 155 ranges {4128768=212992} 2024-09-15T01:45:11.553 INFO:tasks.rados.rados.0.smithi033.stdout:382: writing smithi03327189-4 from 4128768 to 4341760 tid 1 2024-09-15T01:45:11.554 INFO:tasks.rados.rados.0.smithi033.stdout:383: delete oid 21 current snap is 31 2024-09-15T01:45:11.561 INFO:tasks.rados.rados.0.smithi033.stdout:383: done (1 left) 2024-09-15T01:45:11.561 INFO:tasks.rados.rados.0.smithi033.stdout:384: rollback oid 12 current snap is 31 2024-09-15T01:45:11.561 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 12 to 29 2024-09-15T01:45:11.561 INFO:tasks.rados.rados.0.smithi033.stdout:385: snap_remove snap 30 2024-09-15T01:45:11.563 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:11.559+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:11.625 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.110835799, 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:45:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:45:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.101860899000002, 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:45:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.101793237000003, 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.101767496000001, 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.996497572000003, 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:45:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:45:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.017188960999999, 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:45:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:45:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.017055496000001, 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:45:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:45:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.096364216000001, 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:45:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:45:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.092732784999999, 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:45:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:45:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 20.09265911, 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:45:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:45:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 73/66 ECSubWrite(tid=59, reqid=osd.2.0:59, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.093568871999999, 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011892049999999999, 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5900000000000002e-07 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526791+0000", 2024-09-15T01:45:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1916999999999998e-05 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526801+0000", 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7759999999999992e-06 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526805+0000", 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2570000000000001e-06 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527105+0000", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029965900000000003 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527122+0000", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6993000000000001e-05 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538613+0000", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011490879000000001 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538631+0000", 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8110000000000001e-05 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.091935876000001, 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016700869, 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2e-07 2024-09-15T01:45:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528400+0000", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7503000000000001e-05 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528410+0000", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0137e-05 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528415+0000", 2024-09-15T01:45:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4240000000000001e-06 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535553+0000", 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071381459999999997 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535559+0000", 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5210000000000002e-06 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545040+0000", 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094807480000000006 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545073+0000", 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969999999999998e-05 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 76/53 ECSubWrite(tid=189, reqid=osd.7.0:189, at_version=76'28, trim_to=0'0, roll_forward_to=75'26))", 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.091569587999999, 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013688799999999999, 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528759+0000", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1240999999999999e-05 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528776+0000", 2024-09-15T01:45:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6166000000000001e-05 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528783+0000", 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528831+0000", 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7750000000000002e-05 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528838+0000", 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0230000000000004e-06 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542366+0000", 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013528014999999999 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542427+0000", 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0846e-05 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 76/53 ECSubWrite(tid=145, reqid=osd.4.0:145, at_version=76'94, trim_to=0'0, roll_forward_to=75'92))", 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.089068418, 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012260155, 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531266+0000", 2024-09-15T01:45:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019e-05 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531281+0000", 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4214e-05 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531290+0000", 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9290000000000004e-06 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532672+0000", 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013827100000000001 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532682+0000", 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7960000000000007e-06 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543048+0000", 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010366175 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543499+0000", 2024-09-15T01:45:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045100000000000001 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 76/66 ECSubWrite(tid=67, reqid=osd.2.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=76'21))", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.085527021000001, 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013434306999999999, 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534790+0000", 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8699999999999998e-06 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534798+0000", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534804+0000", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7450000000000001e-06 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535012+0000", 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020819 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535021+0000", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.473e-06 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548169+0000", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013148529000000001 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548215+0000", 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5666e-05 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:807 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.016763222, 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012342736999999999, 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 807 2024-09-15T01:45:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603547+0000", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9380000000000001e-06 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603562+0000", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4431e-05 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603604+0000", 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:45:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603622+0000", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7706e-05 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603719+0000", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7546000000000004e-05 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603728+0000", 2024-09-15T01:45:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608524+0000", 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047951790000000001 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608636+0000", 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011230600000000001 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.609358+0000", 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072179799999999995 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613504+0000", 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041461270000000003 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615819+0000", 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023155229999999999 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615887+0000", 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7708999999999998e-05 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:808 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [rollback 30] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.016632412, 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012866819999999999, 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 808 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603677+0000", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603684+0000", 2024-09-15T01:45:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3220000000000004e-06 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603705+0000", 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1056999999999998e-05 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603716+0000", 2024-09-15T01:45:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1073999999999999e-05 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603799+0000", 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3174e-05 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603803+0000", 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.422e-06 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610165+0000", 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006361841 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610363+0000", 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019824 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.611276+0000", 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091360300000000003 2024-09-15T01:45:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613506+0000", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022295459999999998 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616474+0000", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029682599999999999 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616542+0000", 2024-09-15T01:45:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7694999999999995e-05 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:809 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.016522441999999, 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015369404999999999, 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 809 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603787+0000", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603793+0000", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.951e-06 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603822+0000", 2024-09-15T01:45:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8274000000000001e-05 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603833+0000", 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1243000000000001e-05 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603863+0000", 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0425000000000001e-05 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603867+0000", 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2030000000000001e-06 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612025+0000", 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081579340000000004 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612192+0000", 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016787099999999999 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613063+0000", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087049600000000001 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615910+0000", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028468719999999999 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619073+0000", 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031635119999999998 2024-09-15T01:45:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619154+0000", 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1206999999999997e-05 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=159, reqid=client.4632.0:823, at_version=83'106, trim_to=0'0, roll_forward_to=80'103))", 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0689856070000001, 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011255777999999999, 2024-09-15T01:45:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551323+0000", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2160000000000001e-06 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551804+0000", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048123799999999999 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551819+0000", 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551827+0000", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6499999999999996e-06 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551880+0000", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3767999999999997e-05 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551943+0000", 2024-09-15T01:45:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2779000000000003e-05 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562559+0000", 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010615820999999999 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562578+0000", 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:45:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=169, reqid=client.4632.0:845, at_version=83'112, trim_to=0'0, roll_forward_to=83'110))", 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.9862021789999993, 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012514281, 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634107+0000", 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.31e-06 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634298+0000", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019108600000000001 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634338+0000", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300999999999999e-05 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634352+0000", 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3866000000000001e-05 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634422+0000", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0359999999999997e-05 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634432+0000", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5310000000000007e-06 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646573+0000", 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012141322 2024-09-15T01:45:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646620+0000", 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6505000000000003e-05 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:11.729 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:11.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:45:11.731 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:11.723+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:11.831 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:11.827+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:11.841 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:11.841 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:11.841 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:11.841 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:11.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:45:11.933 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:11.930+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:11.951 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:11.951 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:11.951 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:11.951 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:11.951 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:11.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:45:12.034 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:12.027+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.067 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 59.543361779000001, 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.068 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.069 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:45:12.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 59.543303432999998, 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.073 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:45:12.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:45:12.076 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 59.543224676000001, 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:45:12.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.081 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.082 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 25.545139422999998, 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.083 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:45:12.084 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:45:12.085 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.086 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.087 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.088 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:773 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 30] snapc 2d=[2a] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.543057896000001, 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026472887000000001, 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 773 2024-09-15T01:45:12.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.090 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518960+0000", 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1769999999999998e-06 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518990+0000", 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519067+0000", 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7348000000000006e-05 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.091 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519085+0000", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8050000000000002e-05 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519275+0000", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018974899999999999 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519561+0000", 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028586399999999999 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.539158+0000", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019596921999999999 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544838+0000", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056797690000000003 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545308+0000", 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00047005099999999998 2024-09-15T01:45:12.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545429+0000", 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121575 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:774 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.54289573, 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025029231999999998, 2024-09-15T01:45:12.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 774 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.095 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519121+0000", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2330000000000001e-06 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519129+0000", 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7700000000000007e-06 2024-09-15T01:45:12.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519153+0000", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3017e-05 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519164+0000", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1953000000000001e-05 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519853+0000", 2024-09-15T01:45:12.097 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068810500000000005 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519993+0000", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037699999999999 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.543813+0000", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023819931999999999 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544004+0000", 2024-09-15T01:45:12.098 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019107799999999999 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544148+0000", 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014376699999999999 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:821 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~753664] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.5294095030000001, 2024-09-15T01:45:12.099 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024406196000000002, 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 821 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532607+0000", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1720000000000001e-06 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533760+0000", 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001152761 2024-09-15T01:45:12.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533784+0000", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4366e-05 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533795+0000", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1077000000000001e-05 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533897+0000", 2024-09-15T01:45:12.102 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000102403 2024-09-15T01:45:12.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.103 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:12.103 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533911+0000", 2024-09-15T01:45:12.103 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3076e-05 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.536711+0000", 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028006160000000001 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537117+0000", 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040575600000000001 2024-09-15T01:45:12.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.539897+0000", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027803419999999999 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552030+0000", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012132830000000001 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.556945+0000", 2024-09-15T01:45:12.105 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049147720000000004 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557011+0000", 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6025000000000006e-05 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:822 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~786432] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.5248266299999997, 2024-09-15T01:45:12.106 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022305604, 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 822 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.107 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537190+0000", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.373e-06 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538247+0000", 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001056981 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538272+0000", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4587e-05 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538284+0000", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1972e-05 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543940+0000", 2024-09-15T01:45:12.109 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056560489999999998 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.544202+0000", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026208399999999999 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.547032+0000", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028303849999999999 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552099+0000", 2024-09-15T01:45:12.110 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005067289 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559392+0000", 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072926639999999999 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559493+0000", 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010122 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:823 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~655360] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.5198255750000005, 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020918839000000002, 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 823 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542192+0000", 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9799999999999998e-06 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543766+0000", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0015748540000000001 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543797+0000", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0658999999999999e-05 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543841+0000", 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4029000000000001e-05 2024-09-15T01:45:12.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548696+0000", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004854985 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548823+0000", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012638100000000001 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.550883+0000", 2024-09-15T01:45:12.115 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002060596 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557375+0000", 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0064918859999999997 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563026+0000", 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056509919999999996 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.116 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563107+0000", 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1477000000000006e-05 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:824 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~114688] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.5181535430000004, 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021066457, 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.117 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 824 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543863+0000", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0669999999999999e-06 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545718+0000", 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001855675 2024-09-15T01:45:12.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545744+0000", 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5227000000000001e-05 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545753+0000", 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3239999999999992e-06 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.120 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552985+0000", 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072324340000000003 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.553163+0000", 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017793400000000001 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.554394+0000", 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001230411 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.121 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557432+0000", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030381689999999999 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564820+0000", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073879770000000004 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564927+0000", 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010723899999999999 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:826 3.0s0 3:08ecf469:::smithi03327189-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.5162297060000007, 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020342466, 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 826 2024-09-15T01:45:12.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.124 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7899999999999999e-07 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545787+0000", 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6980000000000001e-06 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545791+0000", 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1069999999999998e-06 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.125 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545793+0000", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.278e-06 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557250+0000", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011456579 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557367+0000", 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0001172 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.126 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566116+0000", 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087490199999999997 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566127+0000", 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1205e-05 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:855 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 161] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.127 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.4265906089999998, 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.040891745, 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 855 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:12.128 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635425+0000", 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.189e-06 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.129 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635431+0000", 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635447+0000", 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5540999999999999e-05 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635455+0000", 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8159999999999994e-06 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.130 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635541+0000", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.6370000000000001e-05 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635650+0000", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109366 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.663664+0000", 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028014127999999999 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.667487+0000", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038226110000000001 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676257+0000", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087697060000000004 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676315+0000", 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8604000000000003e-05 2024-09-15T01:45:12.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:856 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 9.4262056160000007, 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029427498999999999, 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.133 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 856 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635814+0000", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2710000000000003e-06 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635824+0000", 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9879999999999999e-06 2024-09-15T01:45:12.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635830+0000", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0870000000000003e-06 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635890+0000", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9703000000000001e-05 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635946+0000", 2024-09-15T01:45:12.136 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6369000000000002e-05 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.664989+0000", 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029042743999999999 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665110+0000", 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121499 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.137 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665236+0000", 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594599999999999 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:901 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 49] snapc 36=[36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.472955367, 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029140195000000001, 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.138 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 901 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:12.139 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589062+0000", 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7829999999999999e-06 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589075+0000", 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3696000000000001e-05 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.140 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589110+0000", 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4675999999999998e-05 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589124+0000", 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3977e-05 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589270+0000", 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014596999999999999 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.141 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589474+0000", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020429199999999999 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.606938+0000", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017463523000000002 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612856+0000", 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0059182130000000003 2024-09-15T01:45:12.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618075+0000", 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052193589999999998 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618199+0000", 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012370600000000001 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:902 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 6.4728671359999996, 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023705106, 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 902 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.144 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589148+0000", 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.327e-06 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589157+0000", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.8789999999999992e-06 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589169+0000", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2130000000000001e-05 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589176+0000", 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6139999999999997e-06 2024-09-15T01:45:12.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589740+0000", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056363300000000002 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589833+0000", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3189000000000005e-05 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612431+0000", 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022597926000000001 2024-09-15T01:45:12.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612658+0000", 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00022690899999999999 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612852+0000", 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194499 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:920 3.3s0 3:c5b7678e:::smithi03327189-28:head [copy-from ver 64] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.473936438, 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035195668999999999, 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 920 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.149 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588081+0000", 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588093+0000", 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2293000000000001e-05 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588129+0000", 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5559999999999998e-05 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588143+0000", 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4185999999999999e-05 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588287+0000", 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014458299999999999 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588563+0000", 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00027543100000000003 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.613913+0000", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025350342000000001 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.616276+0000", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002362496 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623181+0000", 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0069057119999999996 2024-09-15T01:45:12.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623273+0000", 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2150000000000004e-05 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:921 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4738397970000001, 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027587708999999998, 2024-09-15T01:45:12.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 921 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.155 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588176+0000", 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588182+0000", 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2999999999999998e-06 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588210+0000", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7872e-05 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588223+0000", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3183999999999999e-05 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588905+0000", 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068206500000000004 2024-09-15T01:45:12.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589055+0000", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014992299999999999 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615540+0000", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026484735999999998 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615634+0000", 2024-09-15T01:45:12.158 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3851999999999995e-05 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615762+0000", 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012850700000000001 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:924 3.cs0 3:3cf1f308:::smithi03327189-35:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4735288510000002, 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023930099999999999, 2024-09-15T01:45:12.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 924 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.160 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588487+0000", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.254e-06 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588494+0000", 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.161 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588512+0000", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8343e-05 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588522+0000", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.03e-05 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.591976+0000", 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034535500000000001 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.592256+0000", 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000280115 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.612415+0000", 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020159355 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:925 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 49] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:12.163 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4734700519999997, 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027142459000000001, 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 925 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:12.164 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588546+0000", 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.42e-06 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.165 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588553+0000", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8800000000000008e-06 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588572+0000", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8966000000000001e-05 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588584+0000", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1199999999999999e-05 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.166 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588679+0000", 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5367999999999999e-05 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588816+0000", 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000137198 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.607033+0000", 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018216392000000001 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.167 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.611398+0000", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0043651899999999997 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615605+0000", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042073969999999999 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615687+0000", 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1447999999999997e-05 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:926 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 5.4733970300000001, 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021973917999999999, 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 926 2024-09-15T01:45:12.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.170 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588618+0000", 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.203e-06 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588624+0000", 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7930000000000003e-06 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588642+0000", 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8114e-05 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.171 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588653+0000", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0730000000000001e-05 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589048+0000", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039461299999999998 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589152+0000", 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000104838 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610379+0000", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021226630999999999 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610491+0000", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000111718 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610591+0000", 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100278 2024-09-15T01:45:12.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.174 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:12.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:45:12.176 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:12.171+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:12.276 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:12.270+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:12.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:45:12.326 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:12.326 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:12.326 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:12.326 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:12.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:45:12.377 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:12.374+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:12.433 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:12.433 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:12.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:45:12.451 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:12.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:12.478 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:12.475+0000 7fdbfb5c7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22090) UID: 0 2024-09-15T01:45:12.494 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:12.512 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:12.523 INFO:tasks.rados.rados.0.smithi033.stdout:385: done (2 left) 2024-09-15T01:45:12.523 INFO:tasks.rados.rados.0.smithi033.stdout:append oid 27 current snap is 31 2024-09-15T01:45:12.523 INFO:tasks.rados.rados.0.smithi033.stdout:382: finishing write tid 1 to smithi03327189-4 2024-09-15T01:45:12.524 INFO:tasks.rados.rados.0.smithi033.stdout:386: seq_num 157 ranges {442368=245760} 2024-09-15T01:45:12.535 INFO:tasks.rados.rados.0.smithi033.stdout:386: writing smithi03327189-27 from 442368 to 688128 tid 1 2024-09-15T01:45:12.539 INFO:tasks.rados.rados.0.smithi033.stdout:382: finishing write tid 2 to smithi03327189-4 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:382: finishing write tid 3 to smithi03327189-4 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 4 v 142 (ObjNum 155 snap 31 seq_num 155) dirty exists 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:382: left oid 4 (ObjNum 155 snap 31 seq_num 155) 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:384: finishing rollback tid 0 to smithi03327189-12 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:384: finishing rollback tid 1 to smithi03327189-12 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:384: finishing rollback tid 2 to smithi03327189-12 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 12 v 166 (ObjNum 107 snap 13 seq_num 107) dirty exists 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:382: done (2 left) 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:384: done (1 left) 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:387: read oid 49 snap -1 2024-09-15T01:45:12.540 INFO:tasks.rados.rados.0.smithi033.stdout:387: expect deleted 2024-09-15T01:45:12.541 INFO:tasks.rados.rados.0.smithi033.stdout:388: snap_create 2024-09-15T01:45:12.547 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 76.164520218000007, 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006287684, 2024-09-15T01:45:12.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 5 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394850+0000", 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394852+0000", 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7460000000000001e-06 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394870+0000", 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8074e-05 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394905+0000", 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4876000000000003e-05 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.394923+0000", 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7691000000000001e-05 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395069+0000", 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014603000000000001 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.395085+0000", 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6143000000000001e-05 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398289+0000", 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.003203759 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398489+0000", 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000200223 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:45:12.568 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.398670+0000", 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000180359 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.400465+0000", 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001795718 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401043+0000", 2024-09-15T01:45:12.569 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057742000000000004 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401080+0000", 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7283000000000002e-05 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:56.401137+0000", 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7361999999999997e-05 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4317.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 75.149622808000004, 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0068396289999999998, 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4317", 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/2459099544", 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 7 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.571 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409747+0000", 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409750+0000", 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0699999999999998e-06 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409763+0000", 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.325e-05 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409805+0000", 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1412000000000003e-05 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.573 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.409839+0000", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3692999999999997e-05 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410004+0000", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016576300000000001 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.410032+0000", 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8076e-05 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413650+0000", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0036180219999999998 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.413889+0000", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00023881400000000001 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for subops from 1", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.414266+0000", 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00037692599999999998 2024-09-15T01:45:12.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op_commit", 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.415749+0000", 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001483301 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_commit_rec", 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416329+0000", 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057982400000000005 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.576 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416478+0000", 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014846299999999999 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:43:57.416587+0000", 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109015 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:222 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:45:12.577 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.041110473000003, 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026140312999999998, 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 222 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:45:12.578 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518259+0000", 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518262+0000", 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1090000000000001e-06 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.579 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518270+0000", 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.977e-06 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518286+0000", 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6081000000000001e-05 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518298+0000", 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1941e-05 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.580 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518644+0000", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034667699999999999 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518650+0000", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4380000000000003e-06 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526775+0000", 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081258210000000001 2024-09-15T01:45:12.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.526967+0000", 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000191991 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.544400+0000", 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017432277999999999 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.583 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:223 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.583 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:45:12.583 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.041048560999997, 2024-09-15T01:45:12.583 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027026921999999998, 2024-09-15T01:45:12.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.584 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.584 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 223 2024-09-15T01:45:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518321+0000", 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518322+0000", 2024-09-15T01:45:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:45:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518328+0000", 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.4210000000000003e-06 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518338+0000", 2024-09-15T01:45:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.022e-05 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518343+0000", 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.66e-06 2024-09-15T01:45:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518713+0000", 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00036931399999999998 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518715+0000", 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.841e-06 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527539+0000", 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0088243579999999992 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527752+0000", 2024-09-15T01:45:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000212888 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.545348+0000", 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017596414000000001 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:224 3.5s0 3:a51da153:::smithi03327189-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.041015608999999, 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028715145000000001, 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 224 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518354+0000", 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518355+0000", 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.3499999999999996e-07 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518359+0000", 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.9550000000000002e-06 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518367+0000", 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.5920000000000001e-06 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518372+0000", 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.8420000000000004e-06 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518770+0000", 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039736300000000002 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518772+0000", 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2019999999999998e-06 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528208+0000", 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0094362790000000005 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528403+0000", 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019513399999999999 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.599 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.547069+0000", 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018666242999999999 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.9s4 33/32 ECSubWrite(tid=0, reqid=client.4632.0:218, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T01:45:12.600 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.031818858999998, 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0062097970000000004, 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.601 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527551+0000", 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527552+0000", 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.5199999999999996e-07 2024-09-15T01:45:12.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528961+0000", 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0014095609999999999 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528973+0000", 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1335000000000001e-05 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.603 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528990+0000", 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7359000000000001e-05 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529050+0000", 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0235000000000002e-05 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.529062+0000", 2024-09-15T01:45:12.604 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1531e-05 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533731+0000", 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0046689110000000004 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533761+0000", 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0113000000000001e-05 2024-09-15T01:45:12.605 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/32 ECSubWriteReply(tid=61, last_complete=33'22, committed=1, applied=1))", 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 60.031373055000003, 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0050195400000000003, 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527997+0000", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.527998+0000", 2024-09-15T01:45:12.607 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0839999999999999e-06 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528005+0000", 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7209999999999997e-06 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528018+0000", 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3818000000000001e-05 2024-09-15T01:45:12.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.528026+0000", 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1359999999999999e-06 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533000+0000", 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049747359999999996 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.609 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.533016+0000", 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6045e-05 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:255 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.03871401, 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049454870000000001, 2024-09-15T01:45:12.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 255 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520656+0000", 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520660+0000", 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.6600000000000001e-06 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520673+0000", 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3406e-05 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.613 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520707+0000", 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.3902999999999997e-05 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520727+0000", 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0588999999999999e-05 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520839+0000", 2024-09-15T01:45:12.614 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00011144199999999999 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.521184+0000", 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034537000000000001 2024-09-15T01:45:12.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.525601+0000", 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044171169999999999 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:256 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.038595305999998, 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061411549999999997, 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.617 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 256 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520775+0000", 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520776+0000", 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5e-06 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520786+0000", 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9930000000000007e-06 2024-09-15T01:45:12.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520807+0000", 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1169999999999999e-05 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520819+0000", 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1589e-05 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522420+0000", 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001600953 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522494+0000", 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.4438999999999997e-05 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.621 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.526916+0000", 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0044215110000000004 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:257 3.0s0 3:0899e339:::smithi03327189-38:head [read 0~622592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 54.038535211000003, 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.006854008, 2024-09-15T01:45:12.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 257 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520835+0000", 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520836+0000", 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.624 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.625 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520844+0000", 2024-09-15T01:45:12.625 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8599999999999993e-06 2024-09-15T01:45:12.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520862+0000", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.7859999999999998e-05 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.520872+0000", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0438999999999999e-05 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522823+0000", 2024-09-15T01:45:12.626 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0019506020000000001 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.522947+0000", 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012457899999999999 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:18.527689+0000", 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004741398 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(osd.1.17:1 3.0s0 3:08c156cd:::smithi03327189-34:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 50.033441912999997, 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012067648, 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525928+0000", 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525932+0000", 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.693e-06 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.629 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525943+0000", 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0917000000000001e-05 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525964+0000", 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0993e-05 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.525975+0000", 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1823e-05 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.630 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527736+0000", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001760345 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.527967+0000", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000230769 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.537996+0000", 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.010029108 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.631 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=50, reqid=client.4632.0:290, at_version=43'17, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 50.026177945999997, 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061827899999999996, 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.632 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533192+0000", 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.68e-07 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.633 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533416+0000", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000223594 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.533428+0000", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2649e-05 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.534981+0000", 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001553029 2024-09-15T01:45:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535038+0000", 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6560000000000001e-05 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535050+0000", 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1885999999999999e-05 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539340+0000", 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004289836 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539375+0000", 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5067999999999997e-05 2024-09-15T01:45:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWrite(3.1s4 43/34 ECSubWrite(tid=51, reqid=client.4632.0:291, at_version=43'18, trim_to=0'0, roll_forward_to=40'16))", 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 50.024336771999998, 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049865860000000003, 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535033+0000", 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9200000000000002e-07 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535160+0000", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000126678 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535171+0000", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0783e-05 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535178+0000", 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9809999999999997e-06 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535795+0000", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00061714799999999998 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.535804+0000", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.0680000000000003e-06 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.539989+0000", 2024-09-15T01:45:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041847079999999997 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:22.540020+0000", 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0927999999999999e-05 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:393 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 0] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:45:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 46.052208245000003, 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077679840000000003, 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 393 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507162+0000", 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507165+0000", 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2729999999999998e-06 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507179+0000", 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4344e-05 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507215+0000", 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5509000000000002e-05 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507230+0000", 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5116000000000001e-05 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507374+0000", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014417100000000001 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507400+0000", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.6222000000000001e-05 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511249+0000", 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038487730000000002 2024-09-15T01:45:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511443+0000", 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194234 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.514930+0000", 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034863419999999999 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:394 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 46.052115006999998, 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0081504530000000002, 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 394 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507255+0000", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507256+0000", 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1230000000000001e-06 2024-09-15T01:45:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507260+0000", 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.3970000000000004e-06 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507278+0000", 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8077e-05 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507285+0000", 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.9859999999999997e-06 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507540+0000", 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000254541 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.507553+0000", 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2962999999999999e-05 2024-09-15T01:45:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511752+0000", 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0041991509999999999 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.511891+0000", 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013930299999999999 2024-09-15T01:45:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515075+0000", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0031832549999999999 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515236+0000", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000161096 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:26.515405+0000", 2024-09-15T01:45:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00016956100000000001 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:396 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~671744] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.042841316999997, 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.014871097, 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 396 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516529+0000", 2024-09-15T01:45:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.516531+0000", 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8700000000000001e-06 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517417+0000", 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00088526499999999999 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517445+0000", 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8351e-05 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517458+0000", 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2894000000000001e-05 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517618+0000", 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000160025 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.517871+0000", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025296800000000001 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.520096+0000", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0022246259999999999 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523463+0000", 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0033674690000000001 2024-09-15T01:45:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531316+0000", 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0078531639999999993 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531400+0000", 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.3465000000000001e-05 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:397 3.5s0 3:ac2d3331:::smithi03327189-18:head [append 0~770048] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.035493860999999, 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0082945069999999996, 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 397 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523876+0000", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.523879+0000", 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.2610000000000002e-06 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525106+0000", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266060000000001 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525135+0000", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.8670000000000002e-05 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525148+0000", 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3892e-05 2024-09-15T01:45:12.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525342+0000", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019338100000000001 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525610+0000", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026807099999999999 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.528197+0000", 2024-09-15T01:45:12.662 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025871959999999999 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531425+0000", 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0032278430000000002 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532119+0000", 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00069357900000000005 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.663 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532171+0000", 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2008000000000001e-05 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:398 3.5s0 3:ac2d3331:::smithi03327189-18:head [setxattr _header (57) in=7b] snapc 13=[13,12] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.034200480000003, 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073765130000000003, 2024-09-15T01:45:12.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 398 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.665 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525169+0000", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525171+0000", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525178+0000", 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1239999999999999e-06 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525193+0000", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.535e-05 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525203+0000", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.4749999999999999e-06 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530018+0000", 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0048148799999999997 2024-09-15T01:45:12.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530132+0000", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000114434 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.530708+0000", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00057595400000000005 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531935+0000", 2024-09-15T01:45:12.668 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0012266790000000001 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532465+0000", 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00052987900000000003 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532546+0000", 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1151999999999998e-05 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:399 3.5s0 3:ac2d3331:::smithi03327189-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.033846009999998, 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0090256980000000004, 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 399 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.670 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525524+0000", 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.671 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525525+0000", 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5009999999999999e-06 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525531+0000", 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.1120000000000001e-06 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525550+0000", 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8819000000000001e-05 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525559+0000", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.833e-06 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531733+0000", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0061738590000000003 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531869+0000", 2024-09-15T01:45:12.673 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00013559600000000001 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.532982+0000", 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0011133320000000001 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.533176+0000", 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019444500000000001 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.674 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.534550+0000", 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0013732009999999999 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 48/34 ECSubWriteReply(tid=157, last_complete=48'57, committed=1, applied=1))", 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 45.033772049, 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005543699, 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:12.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525598+0000", 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:12.676 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525599+0000", 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0950000000000001e-06 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525606+0000", 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.7349999999999999e-06 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525621+0000", 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5559000000000001e-05 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.677 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.525628+0000", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6560000000000003e-06 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531120+0000", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0054919829999999998 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:27.531142+0000", 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1671e-05 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.678 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:12.679 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:12.679 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:12.679 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:45:12.681 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:12.678+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:12.681 INFO:tasks.rados.rados.0.smithi033.stdout:386: finishing write tid 1 to smithi03327189-27 2024-09-15T01:45:12.681 INFO:tasks.rados.rados.0.smithi033.stdout:386: finishing write tid 2 to smithi03327189-27 2024-09-15T01:45:12.681 INFO:tasks.rados.rados.0.smithi033.stdout:386: finishing write tid 3 to smithi03327189-27 2024-09-15T01:45:12.682 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 27 v 25 (ObjNum 157 snap 31 seq_num 157) dirty exists 2024-09-15T01:45:12.682 INFO:tasks.rados.rados.0.smithi033.stdout:386: left oid 27 (ObjNum 157 snap 31 seq_num 157) 2024-09-15T01:45:12.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:45:12.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:12.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:12.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:12.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:12.683 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:12.687 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:12.718 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:12.722 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:12.783 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:12.775+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:12.843 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:12.843 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:12.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:45:12.882 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:12.879+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:12.953 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:12.954 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:12.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:45:13.005 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:13.003+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:13.084 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:13.079+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:13.105 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.105 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:45:13.186 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:13.182+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:13.219 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.219 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:45:13.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:13.279+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:13.330 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.331 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:45:13.387 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:13.383+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:13.441 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.441 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:45:13.489 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:13.483+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:386: done (2 left) 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:387: done (1 left) 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:388: done (0 left) 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:389: rollback oid 23 current snap is 32 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:rollback oid 23 to 29 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:390: copy_from oid 6 from oid 5 current snap is 32 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:391: read oid 22 snap 29 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:391: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-15T01:45:13.537 INFO:tasks.rados.rados.0.smithi033.stdout:392: read oid 32 snap 28 2024-09-15T01:45:13.538 INFO:tasks.rados.rados.0.smithi033.stdout:392: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:45:13.538 INFO:tasks.rados.rados.0.smithi033.stdout:393: read oid 15 snap -1 2024-09-15T01:45:13.538 INFO:tasks.rados.rados.0.smithi033.stdout:393: expect (ObjNum 131 snap 22 seq_num 131) 2024-09-15T01:45:13.538 INFO:tasks.rados.rados.0.smithi033.stdout:394: delete oid 4 current snap is 32 2024-09-15T01:45:13.555 INFO:tasks.rados.rados.0.smithi033.stdout:389: finishing rollback tid 0 to smithi03327189-23 2024-09-15T01:45:13.560 INFO:tasks.rados.rados.0.smithi033.stdout:389: finishing rollback tid 1 to smithi03327189-23 2024-09-15T01:45:13.560 INFO:tasks.rados.rados.0.smithi033.stdout:389: finishing rollback tid 2 to smithi03327189-23 2024-09-15T01:45:13.560 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 23 v 72 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T01:45:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:45:13.576 INFO:tasks.rados.rados.0.smithi033.stdout:392: expect (ObjNum 81 snap 8 seq_num 81) 2024-09-15T01:45:13.576 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.576 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.589 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:45:13.590 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:13.583+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:13.648 INFO:tasks.rados.rados.0.smithi033.stdout:391: expect (ObjNum 102 snap 13 seq_num 102) 2024-09-15T01:45:13.683 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.683 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.690 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:13.687+0000 7fc4cd4a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22087) UID: 0 2024-09-15T01:45:13.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:45:13.695 INFO:tasks.rados.rados.0.smithi033.stdout:390: finishing copy_from racing read to smithi03327189-6 2024-09-15T01:45:13.695 INFO:tasks.rados.rados.0.smithi033.stdout:393: expect (ObjNum 131 snap 22 seq_num 131) 2024-09-15T01:45:13.788 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.788 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.790 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:13.786+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:13.796 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:45:13.852 INFO:tasks.rados.rados.0.smithi033.stdout:390: finishing copy_from to smithi03327189-6 2024-09-15T01:45:13.852 INFO:tasks.rados.rados.0.smithi033.stdout:update_object_version oid 6 v 148 (ObjNum 134 snap 24 seq_num 134) dirty exists 2024-09-15T01:45:13.852 INFO:tasks.rados.rados.0.smithi033.stdout:389: done (5 left) 2024-09-15T01:45:13.852 INFO:tasks.rados.rados.0.smithi033.stdout:390: done (4 left) 2024-09-15T01:45:13.852 INFO:tasks.rados.rados.0.smithi033.stdout:391: done (3 left) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:392: done (2 left) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:393: done (1 left) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:394: done (0 left) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:395: read oid 49 snap 28 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:395: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:396: rmattr oid 6 current snap is 32 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:396: done (1 left) 2024-09-15T01:45:13.853 INFO:tasks.rados.rados.0.smithi033.stdout:397: snap_create 2024-09-15T01:45:13.870 INFO:tasks.rados.rados.0.smithi033.stdout:395: expect (ObjNum 94 snap 11 seq_num 94) 2024-09-15T01:45:13.892 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:13.890+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:13.898 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:13.898 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:13.907 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:45:13.993 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:13.990+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:14.008 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.009 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:45:14.093 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.087+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:14.121 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.121 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.130 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:45:14.194 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:14.190+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:14.236 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.236 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.245 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:45:14.295 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:14.290+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:14.391 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.392 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.396 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.391+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:14.401 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:45:14.497 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:14.494+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:14.499 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.499 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.508 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:45:14.598 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:14.594+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:14.649 INFO:tasks.rados.rados.0.smithi033.stdout:395: done (1 left) 2024-09-15T01:45:14.649 INFO:tasks.rados.rados.0.smithi033.stdout:397: done (0 left) 2024-09-15T01:45:14.650 INFO:tasks.rados.rados.0.smithi033.stdout:398: snap_create 2024-09-15T01:45:14.658 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.658 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.658 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.658 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.660 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:14.655+0000 7f783ab5b700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.660 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:14.655+0000 7f783ab5b700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.660 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:14.655+0000 7f783ab5b700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.660 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:14.655+0000 7f783ab5b700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.661 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fdbe1b14700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.661 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fdbe1b14700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.661 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fdbe1b14700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.661 INFO:tasks.ceph.osd.6.smithi047.stderr:2024-09-15T01:45:14.655+0000 7fdbe1b14700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.662 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:14.658+0000 7f2af671f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.662 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:14.658+0000 7f2af671f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.662 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:14.658+0000 7f2af671f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.663 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.663 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c808c700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c808c700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c808c700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c808c700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fe4c808c700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15ad91700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15ad91700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15ad91700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15ad91700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.666 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.655+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:14.671 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:45:14.699 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:14.695+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:14.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T01:45:14.762 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.762 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.771 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:45:14.801 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:14.798+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:14.875 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:14.875 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:14.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T01:45:14.901 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:14.899+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:14.912 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:14.934 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:14.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:14.989 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:15.001 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.001 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:15.001 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:15.001 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.002 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:14.995+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:15.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T01:45:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:15.040 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:15.062 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:15.085 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:15.103 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:15.099+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.106 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:15.106 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:15.106 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:15.106 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.109 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:15.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T01:45:15.133 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:15.158 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:15.195 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:15.204 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:15.202+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:15.244 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:210 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~622592] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.712269235000001, 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.028522468, 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 210 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526447+0000", 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526450+0000", 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6510000000000001e-06 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526828+0000", 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000377678 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526856+0000", 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.864e-05 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526866+0000", 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526934+0000", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8579999999999997e-05 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.526938+0000", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5049999999999999e-06 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537847+0000", 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0109092449999999 2024-09-15T01:45:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.538223+0000", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037626800000000002 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.542110+0000", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038865470000000002 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550345+0000", 2024-09-15T01:45:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082352420000000003 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554479+0000", 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041336769999999997 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.554970+0000", 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000491316 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:211 3.1s0 3:85bd41e4:::smithi03327189-14:head [append 0~376832] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.709837837000002, 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0286535569999999, 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 211 2024-09-15T01:45:15.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528879+0000", 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.528880+0000", 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3459999999999999e-06 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529079+0000", 2024-09-15T01:45:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019911399999999999 2024-09-15T01:45:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529096+0000", 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6220000000000001e-05 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529103+0000", 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-06 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529155+0000", 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2358000000000003e-05 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529157+0000", 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9429999999999999e-06 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545244+0000", 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.016087242 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.545568+0000", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032344600000000002 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.546926+0000", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013581789999999999 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550644+0000", 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037184140000000002 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557244+0000", 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065995699999999999 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.557532+0000", 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028843800000000002 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.259 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:212 3.1s0 3:85bd41e4:::smithi03327189-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.709599021999999, 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.030647817, 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 212 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529118+0000", 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8899999999999998e-07 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529123+0000", 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8210000000000001e-06 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529132+0000", 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5199999999999997e-06 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529137+0000", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5060000000000003e-06 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529205+0000", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8025e-05 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529208+0000", 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8650000000000001e-06 2024-09-15T01:45:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.548817+0000", 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.019608812 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549053+0000", 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023625 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.549985+0000", 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093146799999999997 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552463+0000", 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024781880000000001 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559318+0000", 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068554999999999996 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.559766+0000", 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044707299999999998 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:213 3.1s0 3:85bd41e4:::smithi03327189-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.709567638999999, 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0369497130000001, 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 213 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529149+0000", 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529150+0000", 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4700000000000001e-07 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529152+0000", 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3379999999999999e-06 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529160+0000", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1340000000000004e-06 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529165+0000", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5090000000000004e-06 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529247+0000", 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2853000000000001e-05 2024-09-15T01:45:15.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.529248+0000", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1200000000000002e-07 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550397+0000", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0211483210000001 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550639+0000", 2024-09-15T01:45:15.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024213500000000001 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.560762+0000", 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010123007 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.561020+0000", 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025835099999999998 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.566099+0000", 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050787059999999997 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:215 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 62.730360208999997, 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.054175775000000002, 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 215 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508357+0000", 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.508359+0000", 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9980000000000002e-06 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516182+0000", 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078235639999999999 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516197+0000", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516250+0000", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3337000000000001e-05 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516512+0000", 2024-09-15T01:45:15.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026182100000000003 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516516+0000", 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1330000000000001e-06 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550681+0000", 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034164679000000003 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.550941+0000", 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026011699999999998 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.562532+0000", 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01159147 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.277 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:216 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 62.722445807, 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.047031213000000002, 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 216 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516271+0000", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516273+0000", 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9080000000000002e-06 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516279+0000", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3350000000000003e-06 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516289+0000", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8290000000000003e-06 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516294+0000", 2024-09-15T01:45:15.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4750000000000001e-06 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517022+0000", 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072740800000000003 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517024+0000", 2024-09-15T01:45:15.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3369999999999998e-06 2024-09-15T01:45:15.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.551931+0000", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.034907176999999998 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552129+0000", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019750200000000001 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.563302+0000", 2024-09-15T01:45:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011173242 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:217 3.1s0 3:851a3c92:::smithi03327189-1:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 62.722416543999998, 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.048124894000000001, 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 217 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516300+0000", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516301+0000", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4199999999999996e-07 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516305+0000", 2024-09-15T01:45:15.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9790000000000004e-06 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516311+0000", 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6919999999999996e-06 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.516316+0000", 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.109e-06 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517051+0000", 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073586599999999995 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.517052+0000", 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0299999999999997e-07 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552499+0000", 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.035446812000000001 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.552701+0000", 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020239300000000001 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.564425+0000", 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011723598 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:290 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~802816] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.722841225000003, 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024603182000000001, 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 290 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515875+0000", 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.515878+0000", 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9220000000000001e-06 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516503+0000", 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00062464400000000005 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516540+0000", 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7372000000000001e-05 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516554+0000", 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3735e-05 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516695+0000", 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140891 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.516705+0000", 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0087e-05 2024-09-15T01:45:15.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520630+0000", 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003924688 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.520731+0000", 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010148 2024-09-15T01:45:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526210+0000", 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054787179999999996 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530969+0000", 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047593779999999999 2024-09-15T01:45:15.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540362+0000", 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093926119999999998 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540479+0000", 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116655 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.297 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:291 3.1s0 3:86165cc9:::smithi03327189-11:head [append 0~376832] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:15.298 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.720602763999999, 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022692905999999999, 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 291 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518114+0000", 2024-09-15T01:45:15.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518116+0000", 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9379999999999999e-06 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518604+0000", 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000487777 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518640+0000", 2024-09-15T01:45:15.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5932000000000001e-05 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518654+0000", 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4035e-05 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518842+0000", 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000188266 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518855+0000", 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-05 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528128+0000", 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092723119999999996 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528231+0000", 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010355499999999999 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529203+0000", 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00097167800000000002 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.531598+0000", 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023944790000000001 2024-09-15T01:45:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540737+0000", 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091397570000000001 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540807+0000", 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9606999999999999e-05 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:292 3.1s0 3:86165cc9:::smithi03327189-11:head [setxattr _header (56) in=7b] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.720033094999998, 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024528258000000001, 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 292 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518684+0000", 2024-09-15T01:45:15.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518685+0000", 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5179999999999999e-06 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518693+0000", 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9330000000000001e-06 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518717+0000", 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3944999999999999e-05 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518732+0000", 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4592e-05 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518944+0000", 2024-09-15T01:45:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000212208 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518961+0000", 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7666999999999999e-05 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.529935+0000", 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010973966 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530067+0000", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131364 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530602+0000", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00053479700000000005 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.532140+0000", 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001538609 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543132+0000", 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010991393 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.543212+0000", 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0265999999999993e-05 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:293 3.1s0 3:86165cc9:::smithi03327189-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 52.719974127, 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030262566000000001, 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 293 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518743+0000", 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518744+0000", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.043e-06 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518749+0000", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9849999999999997e-06 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518763+0000", 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-15T01:45:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518773+0000", 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3719999999999994e-06 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519047+0000", 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027396899999999999 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.519061+0000", 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4946999999999999e-05 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530880+0000", 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011818222 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.530959+0000", 2024-09-15T01:45:15.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8866e-05 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544741+0000", 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013782345 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544834+0000", 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3420999999999999e-05 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.549005+0000", 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004170811 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:391 3.6s0 3:6579c50e:::smithi03327189-13:head [copy-from ver 4] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.732075919000003, 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016600882000000001, 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 391 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506641+0000", 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506644+0000", 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1930000000000002e-06 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506658+0000", 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4097000000000001e-05 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506688+0000", 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0026e-05 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506706+0000", 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8164000000000001e-05 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506888+0000", 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181235 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.506912+0000", 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4130000000000001e-05 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511049+0000", 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041373099999999999 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.511182+0000", 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013255299999999999 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.517679+0000", 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064972169999999996 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.520372+0000", 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026929839999999998 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523125+0000", 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027533750000000002 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:26.523242+0000", 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116598 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:753 3.1s0 3:86165cc9:::smithi03327189-11:head [copy-from ver 13] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.637365138, 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022114891000000001, 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 753 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601352+0000", 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601354+0000", 2024-09-15T01:45:15.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6000000000000001e-06 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601366+0000", 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601396+0000", 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9901999999999999e-05 2024-09-15T01:45:15.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601406+0000", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0550000000000001e-05 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601519+0000", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112546 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601677+0000", 2024-09-15T01:45:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015800100000000001 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.614051+0000", 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012374232000000001 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617994+0000", 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039432089999999996 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623415+0000", 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054204240000000001 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.623466+0000", 2024-09-15T01:45:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1972000000000003e-05 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:754 3.1s0 3:86165cc9:::smithi03327189-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:15.339 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.637285491, 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015926781000000001, 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.340 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 754 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601431+0000", 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601433+0000", 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4270000000000001e-06 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601437+0000", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2899999999999996e-06 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601452+0000", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4939999999999999e-05 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601461+0000", 2024-09-15T01:45:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7269999999999996e-06 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.601979+0000", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051790899999999999 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.602057+0000", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8237000000000003e-05 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617204+0000", 2024-09-15T01:45:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015147361999999999 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617280+0000", 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6318999999999999e-05 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.617358+0000", 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7570000000000004e-05 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:915 3.6s0 3:6e27a4f5:::smithi03327189-22:head [copy-from ver 13] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6508971369999994, 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022788083000000001, 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 915 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587820+0000", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587822+0000", 2024-09-15T01:45:15.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6639999999999998e-06 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587838+0000", 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5894000000000001e-05 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587878+0000", 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9827999999999999e-05 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587893+0000", 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4538999999999999e-05 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588032+0000", 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000139982 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588210+0000", 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017786399999999999 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.602185+0000", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013974268999999999 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.605186+0000", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030014400000000002 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.610512+0000", 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053261050000000002 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.610608+0000", 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5497999999999999e-05 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:916 3.6s0 3:6e27a4f5:::smithi03327189-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6507586249999999, 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017108080000000001, 2024-09-15T01:45:15.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 916 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587958+0000", 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587959+0000", 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1400000000000001e-06 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587965+0000", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2719999999999997e-06 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587979+0000", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4554e-05 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587989+0000", 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0196e-05 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588496+0000", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050668000000000002 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7958999999999999e-05 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.604843+0000", 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016299200999999999 2024-09-15T01:45:15.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.604955+0000", 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112382 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.605066+0000", 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110696 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:965 3.6s0 3:6798eba1:::smithi03327189-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910254+0000", 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3284622720000003, 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016836904, 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 965 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.358 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910254+0000", 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910254+0000", 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910256+0000", 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1230000000000001e-06 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910260+0000", 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8729999999999998e-06 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910278+0000", 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7654000000000001e-05 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910288+0000", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5619999999999993e-06 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.913334+0000", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030460909999999999 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.913421+0000", 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6978000000000006e-05 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.923159+0000", 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097379489999999992 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.923224+0000", 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5055999999999995e-05 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.927091+0000", 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038676180000000002 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:966 3.6s0 3:6e27a4f5:::smithi03327189-22:head [append 0~0] snapc 38=[38,37] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910295+0000", 2024-09-15T01:45:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3284218030000003, 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016136076999999999, 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 966 2024-09-15T01:45:15.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910295+0000", 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910295+0000", 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910296+0000", 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3900000000000004e-07 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910299+0000", 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.083e-06 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910314+0000", 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5699999999999999e-05 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910320+0000", 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9479999999999996e-06 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.913476+0000", 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031555120000000001 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.913569+0000", 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3647999999999995e-05 2024-09-15T01:45:15.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.914339+0000", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076974499999999996 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.917704+0000", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033644149999999999 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.926335+0000", 2024-09-15T01:45:15.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0086317389999999994 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.926431+0000", 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5648000000000003e-05 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.370 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:967 3.6s0 3:6e27a4f5:::smithi03327189-22:head [rollback 38] snapc 38=[38,37] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910326+0000", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3283904360000003, 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018675537999999998, 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 967 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910326+0000", 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910326+0000", 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910327+0000", 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0800000000000005e-07 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910334+0000", 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7139999999999996e-06 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910346+0000", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2428e-05 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910351+0000", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4360000000000001e-06 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.914963+0000", 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046111939999999999 2024-09-15T01:45:15.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.915063+0000", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100721 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.915541+0000", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00047801199999999999 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.917706+0000", 2024-09-15T01:45:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002164296 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.928959+0000", 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011253776 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.929002+0000", 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2453000000000002e-05 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:968 3.6s0 3:6e27a4f5:::smithi03327189-22:head [append 0~0] snapc 38=[38,37] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910357+0000", 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3283601130000005, 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018811083999999999, 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 968 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910357+0000", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910357+0000", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910357+0000", 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1699999999999998e-07 2024-09-15T01:45:15.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910360+0000", 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5270000000000001e-06 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910378+0000", 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7864999999999999e-05 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910384+0000", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1700000000000002e-06 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.916788+0000", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064043679999999997 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.916907+0000", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011883 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.917417+0000", 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050967700000000005 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.922416+0000", 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049991519999999998 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.929096+0000", 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006679941 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.929168+0000", 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2037000000000002e-05 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.385 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T01:45:15.387 INFO:tasks.ceph.osd.10.smithi155.stderr:2024-09-15T01:45:15.382+0000 7f2b151dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22352) UID: 0 2024-09-15T01:45:15.488 INFO:tasks.ceph.osd.11.smithi155.stderr:2024-09-15T01:45:15.486+0000 7f1c1a27b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22355) UID: 0 2024-09-15T01:45:15.535 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.535 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:15.535 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:15.536 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T01:45:15.590 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:15.587+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:15.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T01:45:15.646 INFO:tasks.rados.rados.0.smithi033.stdout:398: done (0 left) 2024-09-15T01:45:15.646 INFO:tasks.rados.rados.0.smithi033.stdout:399: rmattr oid 11 current snap is 34 2024-09-15T01:45:15.646 INFO:tasks.rados.rados.0.smithi033.stdout:399: done (0 left) 2024-09-15T01:45:15.649 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.649 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:15.649 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:15.649 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:15.649 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.661 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 8, 9, 10] out_osds: [11, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T01:45:15.661 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T01:45:15.661 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T01:45:15.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4af1ec700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.664 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4af1ec700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.665 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:15.659+0000 7fc4af1ec700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.665 INFO:tasks.rados.rados.0.smithi033.stderr:0 errors. 2024-09-15T01:45:15.666 INFO:tasks.rados.rados.0.smithi033.stderr: 2024-09-15T01:45:15.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:45:15.669 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:15.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.669 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:15.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.669 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:15.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.670 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:15.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T01:45:15.671 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.671 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa156d89700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa156d89700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa156d89700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fa156d89700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4cc094700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4cc094700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4cc094700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4cc094700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4cc094700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.676 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:15.663+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:15.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:15.711+0000 7fb3d528a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22075) UID: 0 2024-09-15T01:45:15.791 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:15.783+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:230 3.8s1 3:1768942e:::smithi03327189-19:2 [read 0~311296,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.318886949000003, 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016666382, 2024-09-15T01:45:15.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 230 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521056+0000", 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521057+0000", 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0100000000000005e-07 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521060+0000", 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1439999999999999e-06 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521067+0000", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8719999999999996e-06 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.521071+0000", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.639e-06 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527231+0000", 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0061597570000000001 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.527432+0000", 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020059200000000001 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:12.537722+0000", 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290877 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.849 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:252 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.322007128000003, 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017771714000000001, 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 252 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517936+0000", 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517938+0000", 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.193e-06 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517954+0000", 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.56e-05 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517974+0000", 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.037e-05 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.517983+0000", 2024-09-15T01:45:15.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2760000000000006e-06 2024-09-15T01:45:15.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518160+0000", 2024-09-15T01:45:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017719200000000001 2024-09-15T01:45:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518169+0000", 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0159999999999997e-06 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522465+0000", 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0042955709999999998 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.522658+0000", 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000192585 2024-09-15T01:45:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.524512+0000", 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018548149999999999 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527857+0000", 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033446159999999999 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535633+0000", 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077757080000000001 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535708+0000", 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4771999999999995e-05 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:253 3.8s1 3:1086ba31:::smithi03327189-43:head [rollback 6] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:15.857 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.321937595999998, 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018011894000000001, 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 253 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518005+0000", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518006+0000", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-07 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518009+0000", 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2210000000000002e-06 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518018+0000", 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8030000000000003e-06 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518023+0000", 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5009999999999998e-06 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518229+0000", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020609 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518235+0000", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9540000000000001e-06 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525023+0000", 2024-09-15T01:45:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067881119999999998 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525138+0000", 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011558200000000001 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525738+0000", 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000599198 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.531358+0000", 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056202079999999998 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.535961+0000", 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004603607 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.536017+0000", 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5874000000000001e-05 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:254 3.8s1 3:1086ba31:::smithi03327189-43:head [append 0~0] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.321915060000002, 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020091438999999999, 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 254 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518028+0000", 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8599999999999999e-07 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518033+0000", 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518040+0000", 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518044+0000", 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7290000000000002e-06 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518256+0000", 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021181600000000001 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.518261+0000", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4790000000000004e-06 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526322+0000", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080613160000000007 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.526485+0000", 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016231199999999999 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527030+0000", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054534799999999997 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.534817+0000", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077866330000000003 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538031+0000", 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003214195 2024-09-15T01:45:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538119+0000", 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8499999999999996e-05 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:294 3.8s1 3:1086ba31:::smithi03327189-43:head [copy-from ver 8] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.321847867999999, 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033214265, 2024-09-15T01:45:15.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 294 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518095+0000", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518098+0000", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4590000000000001e-06 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518107+0000", 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8840000000000006e-06 2024-09-15T01:45:15.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518133+0000", 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5505000000000001e-05 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518146+0000", 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3006e-05 2024-09-15T01:45:15.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518257+0000", 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111211 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518266+0000", 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9840000000000007e-06 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523171+0000", 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049046699999999999 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523280+0000", 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010893200000000001 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.542948+0000", 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019668076 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.546705+0000", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037569700000000001 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551215+0000", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045104289999999998 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.551309+0000", 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4139000000000001e-05 2024-09-15T01:45:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:295 3.8s1 3:1086ba31:::smithi03327189-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.321778342999998, 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026214082999999999, 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 295 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518165+0000", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518169+0000", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2600000000000001e-06 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518178+0000", 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.825e-06 2024-09-15T01:45:15.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518183+0000", 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6140000000000002e-06 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518355+0000", 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000172242 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518365+0000", 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523558+0000", 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051933810000000004 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523612+0000", 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3894999999999998e-05 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544170+0000", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020557542000000002 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544279+0000", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108924 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.544379+0000", 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010001399999999999 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.882 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:299 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.321536264999999, 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016201218, 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 299 2024-09-15T01:45:15.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518407+0000", 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518408+0000", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0440000000000001e-06 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518421+0000", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518437+0000", 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.588e-05 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518443+0000", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0820000000000004e-06 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518543+0000", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9735999999999995e-05 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1664999999999999e-05 2024-09-15T01:45:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525470+0000", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069152520000000002 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.525644+0000", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017394099999999999 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534608+0000", 2024-09-15T01:45:15.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089644089999999996 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:300 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.321482830999997, 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017479125000000002, 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.888 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 300 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518460+0000", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7700000000000004e-07 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518463+0000", 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-15T01:45:15.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518478+0000", 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5034999999999999e-05 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518483+0000", 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9910000000000002e-06 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518633+0000", 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014984500000000001 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518650+0000", 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6603e-05 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526102+0000", 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074521500000000003 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526270+0000", 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016777599999999999 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.535939+0000", 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0096694209999999992 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:301 3.4s0 3:2cc5cf52:::smithi03327189-46:head [read 0~1687552,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 53.321410651999997, 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021801668, 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 301 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.894 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518532+0000", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518533+0000", 2024-09-15T01:45:15.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.37e-07 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518536+0000", 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.444e-06 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518549+0000", 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9060000000000001e-06 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518720+0000", 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017080200000000001 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518736+0000", 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5468000000000002e-05 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526692+0000", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007956131 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.526759+0000", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7467999999999998e-05 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.540334+0000", 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013574736 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.898 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:633 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~458752] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.315559972999999, 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016864583999999998, 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 633 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524383+0000", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524385+0000", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3520000000000001e-06 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524833+0000", 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044806700000000002 2024-09-15T01:45:15.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524856+0000", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2541999999999999e-05 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.524869+0000", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2812e-05 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525107+0000", 2024-09-15T01:45:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023854799999999999 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.525123+0000", 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5338000000000001e-05 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527635+0000", 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025122399999999998 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.527867+0000", 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023166199999999999 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.531663+0000", 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037967470000000001 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.536590+0000", 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049263179999999998 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541172+0000", 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045827369999999999 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541248+0000", 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5221000000000004e-05 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.905 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:634 3.8s1 3:1768942e:::smithi03327189-19:head [append 0~507904] snapc 1d=[1c] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.310643151000001, 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016354641999999999, 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 634 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529300+0000", 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529302+0000", 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.542e-06 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.529496+0000", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000193489 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530970+0000", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014745349999999999 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.530981+0000", 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1088e-05 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534768+0000", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037869790000000002 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.534884+0000", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000115921 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.535999+0000", 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001114709 2024-09-15T01:45:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.541070+0000", 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050704779999999998 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545613+0000", 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045431170000000002 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:41.545654+0000", 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783999999999998e-05 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s3 73/53 ECSubWrite(tid=132, reqid=osd.4.0:132, at_version=73'84, trim_to=0'0, roll_forward_to=72'82))", 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.323057181999999, 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015282341, 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516886+0000", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0299999999999998e-07 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516946+0000", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9145999999999999e-05 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516976+0000", 2024-09-15T01:45:15.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0236e-05 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.516993+0000", 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7609999999999999e-05 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517168+0000", 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017460800000000001 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.517197+0000", 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8813e-05 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532115+0000", 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014917629 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.532168+0000", 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3596e-05 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:740 3.4s0 3:20577364:::smithi03327189-24:head [copy-from ver 0] snapc 27=[27,26] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.320373879999998, 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016305848000000001, 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 740 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519569+0000", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519573+0000", 2024-09-15T01:45:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-06 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519590+0000", 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6869999999999999e-05 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519639+0000", 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9432999999999999e-05 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519658+0000", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9083e-05 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521437+0000", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017789539999999999 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521452+0000", 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4952e-05 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530454+0000", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.009001963 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530578+0000", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012395599999999999 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535875+0000", 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052969169999999999 2024-09-15T01:45:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:741 3.4s0 3:20577364:::smithi03327189-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.320196591999999, 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016411578999999999, 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 741 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519746+0000", 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519748+0000", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.04e-06 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519758+0000", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5929999999999995e-06 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519795+0000", 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6860000000000003e-05 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.519810+0000", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5082999999999999e-05 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521558+0000", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017476 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.521568+0000", 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0344e-05 2024-09-15T01:45:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530788+0000", 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092198539999999995 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.530875+0000", 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7698e-05 2024-09-15T01:45:15.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535967+0000", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918409999999997 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536066+0000", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8281000000000003e-05 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.536158+0000", 2024-09-15T01:45:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2385000000000004e-05 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/53 ECSubWrite(tid=183, reqid=client.4632.0:738, at_version=73'21, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.316403148999999, 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018357772000000001, 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523540+0000", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523541+0000", 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.24e-06 2024-09-15T01:45:15.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523599+0000", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7500999999999997e-05 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523628+0000", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9193999999999999e-05 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.523640+0000", 2024-09-15T01:45:15.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2276000000000001e-05 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531608+0000", 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0079682079999999992 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.531619+0000", 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0293e-05 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541866+0000", 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010247114999999999 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.541898+0000", 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1945e-05 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.es2 73/34 ECSubWrite(tid=58, reqid=osd.2.0:58, at_version=73'43, trim_to=0'0, roll_forward_to=72'41))", 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.313663825999999, 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015869550999999999, 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526279+0000", 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526280+0000", 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999996e-07 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526306+0000", 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6282e-05 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526322+0000", 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5977999999999998e-05 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526330+0000", 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.191e-06 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526451+0000", 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120526 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526459+0000", 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4109999999999996e-06 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542088+0000", 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015629295000000001 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.542149+0000", 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0247000000000001e-05 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:811 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~737280] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.311560302, 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022983506000000001, 2024-09-15T01:45:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 811 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528383+0000", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.528386+0000", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.292e-06 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529048+0000", 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066185599999999995 2024-09-15T01:45:15.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529087+0000", 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9653999999999997e-05 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529102+0000", 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4931e-05 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529205+0000", 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103047 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.529215+0000", 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7159999999999998e-06 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533610+0000", 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043948049999999999 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533964+0000", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000354235 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.536912+0000", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029481400000000001 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551101+0000", 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014188383000000001 2024-09-15T01:45:15.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551169+0000", 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8662000000000001e-05 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551366+0000", 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019678499999999999 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:812 3.4s0 3:2f034bb0:::smithi03327189-17:head [append 0~770048] snapc 31=[30] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.306277302, 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022084530000000002, 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 812 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533666+0000", 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.533668+0000", 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5969999999999999e-06 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534580+0000", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091206999999999996 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534606+0000", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5573999999999999e-05 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.534627+0000", 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1571000000000001e-05 2024-09-15T01:45:15.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541493+0000", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068657910000000004 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541606+0000", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011307 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.543077+0000", 2024-09-15T01:45:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014709670000000001 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.553529+0000", 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010451313 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555682+0000", 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021530949999999998 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.555750+0000", 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8481999999999999e-05 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:975 3.4s0 3:25aaa82b:::smithi03327189-9:38 [read 0~4653056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910345+0000", 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9295978360000001, 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021453371999999998, 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 975 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910345+0000", 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910345+0000", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910346+0000", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999999e-07 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910350+0000", 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0199999999999996e-06 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910355+0000", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5729999999999998e-06 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910360+0000", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3030000000000001e-06 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.911094+0000", 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073474599999999999 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.911215+0000", 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121051 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.931799+0000", 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020583021 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:976 3.4s0 3:25aaa82b:::smithi03327189-9:38 [read 0~4653056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.910507+0000", 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.9294359649999997, 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024998952000000001, 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 976 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910507+0000", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910507+0000", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910508+0000", 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910512+0000", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6439999999999999e-06 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910521+0000", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2670000000000006e-06 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.910525+0000", 2024-09-15T01:45:15.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0500000000000002e-06 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.919042+0000", 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0085175619999999994 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.919156+0000", 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000114166 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.935506+0000", 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016349519999999999 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:15.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T01:45:15.959 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:15.951+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:15.965 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":96,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:45:15.635077+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:44:29.503950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-15T01:44:05.562069+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":8,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"93","auid":0,"snap_mode":"selfmanaged","snap_seq":64,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"lrcprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:02.963123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:55.269427+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.973689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.365100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.444318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T01:43:50.170791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.879813+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.901906+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.457571+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:58.658046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:00.143314+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T01:43:50.187329+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.e","osds":[11,9,5,4,6,10,0,1]}],"pg_upmap_items":[{"pgid":"3.a","mappings":[{"from":4,"to":9},{"from":7,"to":3},{"from":6,"to":2},{"from":10,"to":1}]}],"pg_temp":[{"pgid":"3.10","osds":[2147483647,9,1,3,0,2147483647,2147483647,7]},{"pgid":"3.11","osds":[2147483647,4,2147483647,0,3,9,1,2147483647]},{"pgid":"3.12","osds":[0,2147483647,3,1,2,2147483647,2147483647,10]},{"pgid":"3.13","osds":[2147483647,0,2147483647,2,6,1,9,2147483647]},{"pgid":"3.14","osds":[4,8,3,2147483647,2147483647,2147483647,2,1]},{"pgid":"3.15","osds":[2147483647,3,2147483647,4,2,6,0,2147483647]},{"pgid":"3.16","osds":[0,2147483647,1,3,8,2147483647,2147483647,6]},{"pgid":"3.17","osds":[0,2147483647,2147483647,3,4,2147483647,2,1]},{"pgid":"3.18","osds":[3,2147483647,1,2,2147483647,2147483647,4,0]},{"pgid":"3.19","osds":[1,3,4,2147483647,6,2147483647,2147483647,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":52,"length":3},{"begin":60,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 96, 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "d386169c-ec95-4076-a6ad-eaab3ccf750b", 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-09-15T01:43:41.737992+0000", 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-09-15T01:45:15.635077+0000", 2024-09-15T01:45:15.976 INFO:teuthology.orchestra.run.smithi033.stdout: "last_up_change": "2024-09-15T01:43:52.350040+0000", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "last_in_change": "2024-09-15T01:44:29.503950+0000", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_num": 5799936, 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_set": [ 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "pglog_hardlimit", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "purged_snapdirs", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_deletes", 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "sortbitwise" 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_version": 6, 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T01:45:15.977 INFO:teuthology.orchestra.run.smithi033.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_snapshot": "", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_max": 3, 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd": 12, 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "require_min_compat_client": "luminous", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min_compat_client": "luminous", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "require_osd_release": "quincy", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "pools": [ 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 1, 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": ".mgr", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:52.349089+0000", 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-15T01:45:15.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool", 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 1, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 1, 2024-09-15T01:45:15.979 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 1, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 1, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 1, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "20", 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:45:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 0, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 0, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:45:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:45:15.982 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "options": { 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_max": 32, 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_min": 1 2024-09-15T01:45:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": {} 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 2, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "rbd", 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:43:54.322526+0000", 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 1, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 2, 2024-09-15T01:45:15.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 1, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 0, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 8, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 8, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 8, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 8, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 8, 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:45:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "23", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 3, 2024-09-15T01:45:15.986 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 23, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:45:15.987 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "", 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:45:15.988 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 0, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd": {} 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": "unique_pool_0", 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "create_time": "2024-09-15T01:44:05.562069+0000", 2024-09-15T01:45:15.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 8193, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": 3, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 8, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": 5, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_rule": 1, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_count": 0, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_target": 0, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "object_hash": 2, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscale_mode": "off", 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": 26, 2024-09-15T01:45:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num": 26, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_placement_num_target": 26, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_target": 26, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num_pending": 26, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "last_pg_merge_meta": { 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "source_pgid": "0.0", 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "ready_epoch": 0, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_started": 0, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "last_epoch_clean": 0, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "source_version": "0'0", 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "target_version": "0'0" 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "last_change": "96", 2024-09-15T01:45:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend": "0", 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "last_force_op_resend_preluminous": "93", 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "auid": 0, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_mode": "selfmanaged", 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_seq": 64, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_epoch": 96, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_snaps": [], 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps": "[]", 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_bytes": 0, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "quota_max_objects": 0, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tiers": [], 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tier_of": -1, 2024-09-15T01:45:15.992 INFO:teuthology.orchestra.run.smithi033.stdout: "read_tier": -1, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "write_tier": -1, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_mode": "none", 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_bytes": 0, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "target_max_objects": 0, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_flush_age": 0, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cache_min_evict_age": 0, 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profile": "lrcprofile", 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_params": { 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "none" 2024-09-15T01:45:15.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_period": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_count": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "use_gmt_hitset": true, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min_read_recency_for_promote": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min_write_recency_for_promote": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "hit_set_search_last_n": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "grade_table": [], 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "stripe_width": 16384, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "expected_num_objects": 0, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "fast_read": false, 2024-09-15T01:45:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "options": {}, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "application_metadata": { 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "rados": {} 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "5bbf5b49-bace-424f-b96c-91c3a25e48e0", 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:15.995 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:15.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-09-15T01:45:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:15.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.000 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6802", 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6803", 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6806", 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6807", 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6804", 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6805", 2024-09-15T01:45:16.003 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22087 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6801/22087", 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6803/22087", 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6807/22087", 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6805/22087", 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464", 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.005 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6808", 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6809", 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6810", 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6811", 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6814", 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6815", 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6812", 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6813", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22086 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6809/22086", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6811/22086", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6815/22086", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6813/22086", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.010 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "fcc939b7-c3fb-4117-af41-9eec226f3020", 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 66, 2024-09-15T01:45:16.011 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6816", 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6817", 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6818", 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6819", 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6822", 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6823", 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6820", 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6821", 2024-09-15T01:45:16.015 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22088 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6817/22088", 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6819/22088", 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6823/22088", 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6821/22088", 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.016 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "57f1d668-f85b-4b0c-9093-fb5fef4b91ef", 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.017 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6824", 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6825", 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6826", 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6827", 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6830", 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6831", 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6828", 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6829", 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22089 2024-09-15T01:45:16.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6825/22089", 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.33:6827/22089", 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.33:6831/22089", 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.33:6829/22089", 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:45:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "a274d12c-8683-491d-8656-5af5d12164ee", 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6800", 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6801", 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6802", 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6803", 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6806", 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6807", 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.026 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6804", 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6805", 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22096 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6801/22096", 2024-09-15T01:45:16.027 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6803/22096", 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6807/22096", 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6805/22096", 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "f0a90a47-9b16-46f6-a2a7-afa4a6f2a298", 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 0, 2024-09-15T01:45:16.028 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 34, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6816", 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6817", 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6818", 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6819", 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6822", 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6823", 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6820", 2024-09-15T01:45:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6821", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22097 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6817/22097", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6819/22097", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6823/22097", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6821/22097", 2024-09-15T01:45:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "194e23d1-7f52-43ee-8557-52d75a3c22dc", 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.034 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6808", 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6809", 2024-09-15T01:45:16.035 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6810", 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6811", 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6814", 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6815", 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6812", 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6813", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22108 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6809/22108", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6811/22108", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6815/22108", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6813/22108", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1d1c61e7-be03-4c72-bc95-e187eb95a1b8", 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 18, 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6824", 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6825", 2024-09-15T01:45:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6826", 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6827", 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6830", 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6831", 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6828", 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.47:6829", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22109 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.47:6825/22109", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.47:6827/22109", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.47:6831/22109", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.47:6829/22109", 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "1f23463d-2214-416b-b3ba-23132f2d84ee", 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 0, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 52, 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6801", 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6803", 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6805", 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6807", 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6813", 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6815", 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6809", 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6811", 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22357 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6803/22357", 2024-09-15T01:45:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6807/22357", 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6815/22357", 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6811/22357", 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "39226fc7-a654-41c6-9d70-5ea2f69c236b", 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 93, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6800", 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6802", 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6804", 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6806", 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6812", 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6814", 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6808", 2024-09-15T01:45:16.056 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6810", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22359 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6802/22359", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6806/22359", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6814/22359", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6810/22359", 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.057 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "eaafc17e-407d-43aa-8083-2c56660d7db4", 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 1, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 1, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.058 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 95, 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6816", 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6817", 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6818", 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6819", 2024-09-15T01:45:16.060 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6822", 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6823", 2024-09-15T01:45:16.061 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6820", 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6821", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22358 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6817/22358", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6819/22358", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6823/22358", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6821/22358", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uuid": "05fdd01d-8c78-4a61-81e7-1d9816e951a3", 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "up": 1, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "in": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_affinity": 1, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_begin": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "last_clean_end": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "up_from": 17, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "up_thru": 27, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "down_at": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "lost_at": 0, 2024-09-15T01:45:16.064 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6824", 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6825", 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.065 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addrs": { 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6826", 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.066 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6827", 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addrs": { 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6830", 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.067 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6831", 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addrs": { 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6828", 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.155:6829", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22360 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.155:6825/22360", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_addr": "172.21.15.155:6827/22360", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_back_addr": "172.21.15.155:6831/22360", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "heartbeat_front_addr": "172.21.15.155:6829/22360", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "state": [ 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "exists", 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "up" 2024-09-15T01:45:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_xinfo": [ 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 0, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:02.963123+0000", 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 1, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:44:55.269427+0000", 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 2, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.071 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:11.973689+0000", 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 3, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.072 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:11.365100+0000", 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 4, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:14.444318+0000", 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 5, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 65536, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.170791+0000", 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 6, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.074 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:14.879813+0000", 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 7, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.075 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:05.901906+0000", 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 8, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:05.457571+0000", 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.076 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 9, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:44:58.658046+0000", 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 10, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 0, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:45:00.143314+0000", 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": 11, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "down_stamp": "0.000000", 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_probability": 0, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "laggy_interval": 0, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "features": 4540138320759226367, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "old_weight": 65536, 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "last_purged_snaps_scrub": "2024-09-15T01:43:50.187329+0000", 2024-09-15T01:45:16.078 INFO:teuthology.orchestra.run.smithi033.stdout: "dead_epoch": 0 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap": [ 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.e", 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 11, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 9, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 5, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 6, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 10, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.079 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_upmap_items": [ 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.a", 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "mappings": [ 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "from": 4, 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "to": 9 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.080 INFO:teuthology.orchestra.run.smithi033.stdout: "from": 7, 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "to": 3 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "from": 6, 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "to": 2 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "from": 10, 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "to": 1 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_temp": [ 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.10", 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 9, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: 7 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.11", 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 9, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.083 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.12", 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: 10 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.084 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.13", 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 6, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 9, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.14", 2024-09-15T01:45:16.085 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 8, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.15", 2024-09-15T01:45:16.086 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 6, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.087 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.16", 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 8, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: 6 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.088 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.17", 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.18", 2024-09-15T01:45:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 2, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: 0 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: "pgid": "3.19", 2024-09-15T01:45:16.090 INFO:teuthology.orchestra.run.smithi033.stdout: "osds": [ 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 3, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 4, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 6, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2147483647, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: "primary_temp": [], 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklist": {}, 2024-09-15T01:45:16.091 INFO:teuthology.orchestra.run.smithi033.stdout: "range_blocklist": {}, 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "erasure_code_profiles": { 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default": { 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "2", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "1", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "jerasure", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "technique": "reed_sol_van" 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "lrcprofile": { 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-device-class": "", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-failure-domain": "osd", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-root": "default", 2024-09-15T01:45:16.092 INFO:teuthology.orchestra.run.smithi033.stdout: "k": "4", 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "l": "3", 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "m": "2", 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "lrcprofile", 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "plugin": "lrc" 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_snaps_queue": [ 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 3, 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "snaps": [ 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.093 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 52, 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 3 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "begin": 60, 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "length": 1 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "new_removed_snaps": [], 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "new_purged_snaps": [], 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_node_flags": {}, 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "device_class_flags": {}, 2024-09-15T01:45:16.094 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": { 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_enabled": false, 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_bucket_count": 0, 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: "degraded_stretch_mode": 0, 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: "recovering_stretch_mode": 0, 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode_bucket": 0 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.095 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:16.099 INFO:tasks.thrashosds.thrasher:j is {'epoch': 96, 'fsid': 'd386169c-ec95-4076-a6ad-eaab3ccf750b', 'created': '2024-09-15T01:43:41.737992+0000', 'modified': '2024-09-15T01:45:15.635077+0000', 'last_up_change': '2024-09-15T01:43:52.350040+0000', 'last_in_change': '2024-09-15T01:44:29.503950+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:43:52.349089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:43:54.322526+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T01:44:05.562069+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 8, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '93', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 64, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'lrcprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '5bbf5b49-bace-424f-b96c-91c3a25e48e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 22087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 22087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 22087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 22087}, {'type': 'v1', 'addr': '172.21.15.33:6805', 'nonce': 22087}]}, 'public_addr': '172.21.15.33:6801/22087', 'cluster_addr': '172.21.15.33:6803/22087', 'heartbeat_back_addr': '172.21.15.33:6807/22087', 'heartbeat_front_addr': '172.21.15.33:6805/22087', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6808', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6809', 'nonce': 22086}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6810', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 22086}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6814', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 22086}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 22086}, {'type': 'v1', 'addr': '172.21.15.33:6813', 'nonce': 22086}]}, 'public_addr': '172.21.15.33:6809/22086', 'cluster_addr': '172.21.15.33:6811/22086', 'heartbeat_back_addr': '172.21.15.33:6815/22086', 'heartbeat_front_addr': '172.21.15.33:6813/22086', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fcc939b7-c3fb-4117-af41-9eec226f3020', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6817', 'nonce': 22088}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6818', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 22088}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 22088}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 22088}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 22088}]}, 'public_addr': '172.21.15.33:6817/22088', 'cluster_addr': '172.21.15.33:6819/22088', 'heartbeat_back_addr': '172.21.15.33:6823/22088', 'heartbeat_front_addr': '172.21.15.33:6821/22088', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57f1d668-f85b-4b0c-9093-fb5fef4b91ef', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6824', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6825', 'nonce': 22089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6826', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6827', 'nonce': 22089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6830', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6831', 'nonce': 22089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6828', 'nonce': 22089}, {'type': 'v1', 'addr': '172.21.15.33:6829', 'nonce': 22089}]}, 'public_addr': '172.21.15.33:6825/22089', 'cluster_addr': '172.21.15.33:6827/22089', 'heartbeat_back_addr': '172.21.15.33:6831/22089', 'heartbeat_front_addr': '172.21.15.33:6829/22089', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a274d12c-8683-491d-8656-5af5d12164ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6800', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6801', 'nonce': 22096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6802', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6803', 'nonce': 22096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6806', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6807', 'nonce': 22096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6804', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.47:6805', 'nonce': 22096}]}, 'public_addr': '172.21.15.47:6801/22096', 'cluster_addr': '172.21.15.47:6803/22096', 'heartbeat_back_addr': '172.21.15.47:6807/22096', 'heartbeat_front_addr': '172.21.15.47:6805/22096', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f0a90a47-9b16-46f6-a2a7-afa4a6f2a298', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6816', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6817', 'nonce': 22097}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6818', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6819', 'nonce': 22097}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6822', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6823', 'nonce': 22097}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6820', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.47:6821', 'nonce': 22097}]}, 'public_addr': '172.21.15.47:6817/22097', 'cluster_addr': '172.21.15.47:6819/22097', 'heartbeat_back_addr': '172.21.15.47:6823/22097', 'heartbeat_front_addr': '172.21.15.47:6821/22097', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '194e23d1-7f52-43ee-8557-52d75a3c22dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6808', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6809', 'nonce': 22108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6810', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6811', 'nonce': 22108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6814', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6815', 'nonce': 22108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6812', 'nonce': 22108}, {'type': 'v1', 'addr': '172.21.15.47:6813', 'nonce': 22108}]}, 'public_addr': '172.21.15.47:6809/22108', 'cluster_addr': '172.21.15.47:6811/22108', 'heartbeat_back_addr': '172.21.15.47:6815/22108', 'heartbeat_front_addr': '172.21.15.47:6813/22108', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '1d1c61e7-be03-4c72-bc95-e187eb95a1b8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6824', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6825', 'nonce': 22109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6826', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6827', 'nonce': 22109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6830', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6831', 'nonce': 22109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6828', 'nonce': 22109}, {'type': 'v1', 'addr': '172.21.15.47:6829', 'nonce': 22109}]}, 'public_addr': '172.21.15.47:6825/22109', 'cluster_addr': '172.21.15.47:6827/22109', 'heartbeat_back_addr': '172.21.15.47:6831/22109', 'heartbeat_front_addr': '172.21.15.47:6829/22109', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '1f23463d-2214-416b-b3ba-23132f2d84ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6801', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6803', 'nonce': 22357}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6805', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6807', 'nonce': 22357}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6813', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6815', 'nonce': 22357}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6809', 'nonce': 22357}, {'type': 'v1', 'addr': '172.21.15.155:6811', 'nonce': 22357}]}, 'public_addr': '172.21.15.155:6803/22357', 'cluster_addr': '172.21.15.155:6807/22357', 'heartbeat_back_addr': '172.21.15.155:6815/22357', 'heartbeat_front_addr': '172.21.15.155:6811/22357', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '39226fc7-a654-41c6-9d70-5ea2f69c236b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6800', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6802', 'nonce': 22359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6804', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6806', 'nonce': 22359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6812', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6814', 'nonce': 22359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6808', 'nonce': 22359}, {'type': 'v1', 'addr': '172.21.15.155:6810', 'nonce': 22359}]}, 'public_addr': '172.21.15.155:6802/22359', 'cluster_addr': '172.21.15.155:6806/22359', 'heartbeat_back_addr': '172.21.15.155:6814/22359', 'heartbeat_front_addr': '172.21.15.155:6810/22359', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'eaafc17e-407d-43aa-8083-2c56660d7db4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6816', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6817', 'nonce': 22358}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6818', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6819', 'nonce': 22358}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6822', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6823', 'nonce': 22358}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6820', 'nonce': 22358}, {'type': 'v1', 'addr': '172.21.15.155:6821', 'nonce': 22358}]}, 'public_addr': '172.21.15.155:6817/22358', 'cluster_addr': '172.21.15.155:6819/22358', 'heartbeat_back_addr': '172.21.15.155:6823/22358', 'heartbeat_front_addr': '172.21.15.155:6821/22358', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '05fdd01d-8c78-4a61-81e7-1d9816e951a3', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6824', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6825', 'nonce': 22360}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6826', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6827', 'nonce': 22360}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6830', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6831', 'nonce': 22360}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6828', 'nonce': 22360}, {'type': 'v1', 'addr': '172.21.15.155:6829', 'nonce': 22360}]}, 'public_addr': '172.21.15.155:6825/22360', 'cluster_addr': '172.21.15.155:6827/22360', 'heartbeat_back_addr': '172.21.15.155:6831/22360', 'heartbeat_front_addr': '172.21.15.155:6829/22360', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:02.963123+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:44:55.269427+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:11.973689+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:11.365100+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:14.444318+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.170791+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:14.879813+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:05.901906+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:05.457571+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:44:58.658046+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T01:45:00.143314+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T01:43:50.187329+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [11, 9, 5, 4, 6, 10, 0, 1]}], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 4, 'to': 9}, {'from': 7, 'to': 3}, {'from': 6, 'to': 2}, {'from': 10, 'to': 1}]}], 'pg_temp': [{'pgid': '3.10', 'osds': [2147483647, 9, 1, 3, 0, 2147483647, 2147483647, 7]}, {'pgid': '3.11', 'osds': [2147483647, 4, 2147483647, 0, 3, 9, 1, 2147483647]}, {'pgid': '3.12', 'osds': [0, 2147483647, 3, 1, 2, 2147483647, 2147483647, 10]}, {'pgid': '3.13', 'osds': [2147483647, 0, 2147483647, 2, 6, 1, 9, 2147483647]}, {'pgid': '3.14', 'osds': [4, 8, 3, 2147483647, 2147483647, 2147483647, 2, 1]}, {'pgid': '3.15', 'osds': [2147483647, 3, 2147483647, 4, 2, 6, 0, 2147483647]}, {'pgid': '3.16', 'osds': [0, 2147483647, 1, 3, 8, 2147483647, 2147483647, 6]}, {'pgid': '3.17', 'osds': [0, 2147483647, 2147483647, 3, 4, 2147483647, 2, 1]}, {'pgid': '3.18', 'osds': [3, 2147483647, 1, 2, 2147483647, 2147483647, 4, 0]}, {'pgid': '3.19', 'osds': [1, 3, 4, 2147483647, 6, 2147483647, 2147483647, 2]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'lrcprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'l': '3', 'm': '2', 'name': 'lrcprofile', 'plugin': 'lrc'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 52, 'length': 3}, {'begin': 60, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T01:45:16.099 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.a 2024-09-15T01:45:16.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.a 2024-09-15T01:45:16.100 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:16.100 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:16.100 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:16.100 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T01:45:16.101 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:16.091+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:16.197 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.194+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:16.240 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:16.241 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:16.241 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:16.241 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:16.241 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T01:45:16.298 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.291+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:258 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 7=[6] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:16.355 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 57.827835069999999, 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016498779000000002, 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 258 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521743+0000", 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521746+0000", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8660000000000002e-06 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521757+0000", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1936999999999999e-05 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521789+0000", 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1223000000000002e-05 2024-09-15T01:45:16.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521806+0000", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7782999999999999e-05 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521942+0000", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135986 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.521961+0000", 2024-09-15T01:45:16.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8352e-05 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525362+0000", 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034012449999999998 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.525522+0000", 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015981200000000001 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.527092+0000", 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015698750000000001 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.530988+0000", 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038960959999999999 2024-09-15T01:45:16.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538163+0000", 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071752079999999998 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:18.538241+0000", 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8396e-05 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.362 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:519 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.699771447000003, 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011551715000000001, 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 519 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649806+0000", 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649807+0000", 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4900000000000005e-07 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649810+0000", 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6230000000000001e-06 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649817+0000", 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.649820+0000", 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8519999999999997e-06 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653088+0000", 2024-09-15T01:45:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032678289999999999 2024-09-15T01:45:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:16.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.653216+0000", 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012757299999999999 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656262+0000", 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030464960000000001 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.656348+0000", 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6147e-05 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:36.661358+0000", 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005009487 2024-09-15T01:45:16.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:563 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~688128] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.598807321999999, 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013708158, 2024-09-15T01:45:16.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 563 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750770+0000", 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.750773+0000", 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0130000000000002e-06 2024-09-15T01:45:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751100+0000", 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00032635200000000002 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751140+0000", 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0666e-05 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751157+0000", 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.611e-05 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751256+0000", 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9264000000000004e-05 2024-09-15T01:45:16.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.751399+0000", 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014319699999999999 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.752929+0000", 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001530317 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764168+0000", 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011238606999999999 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764263+0000", 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5315000000000005e-05 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764479+0000", 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000215317 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:564 3.es0 3:720ccbad:::smithi03327189-29:head [append 0~507904] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 38.595367748000001, 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010646180999999999, 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 564 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754210+0000", 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754212+0000", 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4930000000000001e-06 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.754975+0000", 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076263199999999996 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755006+0000", 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1254000000000003e-05 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755024+0000", 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7836e-05 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755372+0000", 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000347738 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.755682+0000", 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030998499999999998 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.757870+0000", 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021882360000000001 2024-09-15T01:45:16.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764489+0000", 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066186159999999999 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764811+0000", 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000321799 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.764856+0000", 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5592e-05 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:676 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.832424190000001, 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020769342999999999, 2024-09-15T01:45:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 676 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517154+0000", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517156+0000", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-06 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517162+0000", 2024-09-15T01:45:16.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4300000000000003e-06 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517176+0000", 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4335000000000001e-05 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517186+0000", 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2539999999999998e-06 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517261+0000", 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5300000000000001e-05 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517384+0000", 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012365600000000001 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.537923+0000", 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020538379999999998 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:677 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.832373270000001, 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027168877000000001, 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 677 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:16.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517204+0000", 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517205+0000", 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4300000000000002e-07 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517209+0000", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.89e-06 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517217+0000", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0839999999999992e-06 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517222+0000", 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-15T01:45:16.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518136+0000", 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000914269 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.518648+0000", 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00051186899999999997 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544373+0000", 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.025725331000000001 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:678 3.es0 3:720ccbad:::smithi03327189-29:head [read 0~4243456,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.832308260000001, 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030405096999999999, 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 678 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517269+0000", 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517271+0000", 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.141e-06 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517278+0000", 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517316+0000", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8259000000000001e-05 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.517325+0000", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9199999999999993e-06 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521426+0000", 2024-09-15T01:45:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041004759999999996 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.521535+0000", 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109039 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547675+0000", 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026139907 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.817135652000001, 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012132291999999999, 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:16.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.532442+0000", 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.72e-07 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536461+0000", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040183060000000001 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536472+0000", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1455e-05 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536478+0000", 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-15T01:45:16.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538295+0000", 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018168839999999999 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544574+0000", 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062793579999999996 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 67/34 ECSubReadReply(tid=33, attrs_read=0))", 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.813091585999999, 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011243161, 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536486+0000", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.536487+0000", 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2399999999999999e-07 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538222+0000", 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001735728 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538235+0000", 2024-09-15T01:45:16.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2459e-05 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.538242+0000", 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8170000000000001e-06 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.544602+0000", 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063605789999999999 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:46.547729+0000", 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003127154 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:817 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:16.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.809494055, 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012084411, 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 817 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540084+0000", 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540086+0000", 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1380000000000001e-06 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540094+0000", 2024-09-15T01:45:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5210000000000002e-06 2024-09-15T01:45:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540112+0000", 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7937000000000001e-05 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540121+0000", 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0319999999999992e-06 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540573+0000", 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045140399999999998 2024-09-15T01:45:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540597+0000", 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4651000000000002e-05 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546537+0000", 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059398979999999999 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.546648+0000", 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110967 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552168+0000", 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055198629999999999 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:818 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:16.414 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.809436779, 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012454458, 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 818 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540141+0000", 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540142+0000", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540146+0000", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0489999999999997e-06 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540242+0000", 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6168999999999994e-05 2024-09-15T01:45:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540256+0000", 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4063000000000001e-05 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540737+0000", 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048110699999999999 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540764+0000", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7509999999999999e-05 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550905+0000", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010140697000000001 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.550984+0000", 2024-09-15T01:45:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8906000000000007e-05 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552595+0000", 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016113309999999999 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:819 3.as0 3:592fd4c9:::smithi03327189-3:head [read 0~49152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T01:45:16.420 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.809302729000001, 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012574444000000001, 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 819 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:16.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540275+0000", 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540276+0000", 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3570000000000001e-06 2024-09-15T01:45:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540284+0000", 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.605e-06 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540297+0000", 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3526000000000001e-05 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.540304+0000", 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2249999999999997e-06 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541504+0000", 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012007179999999999 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.541512+0000", 2024-09-15T01:45:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1099999999999997e-06 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551300+0000", 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0097880039999999995 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.551387+0000", 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7618e-05 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:01.552849+0000", 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001462281 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:832 3.as0 3:5045eccb:::smithi03327189-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.797230709000001, 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010647103, 2024-09-15T01:45:16.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 832 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552347+0000", 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552348+0000", 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2079999999999999e-06 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552356+0000", 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9300000000000003e-06 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552373+0000", 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6677999999999999e-05 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552381+0000", 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552513+0000", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013210000000000001 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.552519+0000", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9020000000000004e-06 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557555+0000", 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050360889999999997 2024-09-15T01:45:16.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.557647+0000", 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2503000000000005e-05 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562994+0000", 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053469040000000004 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:917 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7618124630000001, 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013596409, 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 917 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.432 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587765+0000", 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587768+0000", 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4480000000000001e-06 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587781+0000", 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2958e-05 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587818+0000", 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7128000000000001e-05 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587833+0000", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5404999999999999e-05 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587920+0000", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6885999999999994e-05 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588036+0000", 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116384 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601362+0000", 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013325200000000001 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:918 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7617127959999994, 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018684564000000001, 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 918 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587865+0000", 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587871+0000", 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1920000000000002e-06 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587878+0000", 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.72e-06 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587893+0000", 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5141e-05 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587901+0000", 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8359999999999992e-06 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588509+0000", 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00060832400000000004 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.588593+0000", 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3774000000000001e-05 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606549+0000", 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017956577000000001 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:919 3.es0 3:7de15fb4:::smithi03327189-49:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:16.441 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7616663670000001, 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.024313863000000002, 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 919 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:16.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587911+0000", 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587912+0000", 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.16e-07 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587917+0000", 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.442e-06 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587925+0000", 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2160000000000006e-06 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.587928+0000", 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6179999999999999e-06 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.592395+0000", 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044660990000000003 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.592821+0000", 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042640199999999999 2024-09-15T01:45:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.612225+0000", 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019404270000000001 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 87/34 ECSubReadReply(tid=90, attrs_read=0))", 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7539180630000004, 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011032053, 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.595660+0000", 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0900000000000002e-07 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601572+0000", 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0059121219999999997 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601581+0000", 2024-09-15T01:45:16.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3690000000000007e-06 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601584+0000", 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8299999999999998e-06 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601881+0000", 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029641800000000001 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606692+0000", 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048108049999999996 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpReadReply(3.es0 87/34 ECSubReadReply(tid=91, attrs_read=0))", 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.7479794210000001, 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010736961, 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:16.451 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601598+0000", 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.601599+0000", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4099999999999997e-07 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606083+0000", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044838739999999997 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606100+0000", 2024-09-15T01:45:16.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6918999999999998e-05 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606107+0000", 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6139999999999997e-06 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.606753+0000", 2024-09-15T01:45:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00064655699999999997 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:06.612335+0000", 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055820560000000002 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:949 3.es0 3:7de15fb4:::smithi03327189-49:head [stat,delete] snapc 38=[38,37] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.718646358, 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013716572, 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 949 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630931+0000", 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630933+0000", 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1050000000000002e-06 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630941+0000", 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9500000000000001e-06 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630965+0000", 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4029e-05 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.630977+0000", 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1163e-05 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.631731+0000", 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00075429399999999997 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.631986+0000", 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025550099999999999 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.632863+0000", 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087646399999999998 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.635373+0000", 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002509945 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.644561+0000", 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0091884740000000003 2024-09-15T01:45:16.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.644648+0000", 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6646999999999999e-05 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s2 91/53 ECSubWrite(tid=159, reqid=osd.9.0:159, at_version=91'184, trim_to=0'0, roll_forward_to=90'180))", 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.7150192400000002, 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011302095, 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634558+0000", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634559+0000", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0599999999999999e-07 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634587+0000", 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7236000000000001e-05 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634598+0000", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1925e-05 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634605+0000", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8859999999999998e-06 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634651+0000", 2024-09-15T01:45:16.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5655000000000002e-05 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.634661+0000", 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0186999999999999e-05 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.645840+0000", 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011179011000000001 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:16.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:10.645861+0000", 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0389000000000001e-05 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T01:45:16.471 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:16.473 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-15T01:45:16.306847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":119226931,"num_objects":66,"num_object_clones":14,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":2,"num_read":517,"num_read_kb":253826,"num_write":935,"num_write_kb":195080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":39703091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1118,"ondisk_log_size":1118,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":10,"kb":943718400,"kb_used":3253684,"kb_used_data":238500,"kb_used_omap":16,"kb_used_meta":3015023,"kb_avail":940464716,"statfs":{"total":966367641600,"available":963035869184,"internally_reserved":0,"allocated":244224000,"data_stored":276720871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3087383920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4669440,"num_objects":-3,"num_object_clones":-3,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":-1,"num_read":5,"num_read_kb":1778,"num_write":2,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001479"},"pg_stats":[{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.615801+0000","last_change":"2024-09-15T01:43:57.903860+0000","last_active":"2024-09-15T01:45:09.615801+0000","last_peered":"2024-09-15T01:45:09.615801+0000","last_clean":"2024-09-15T01:45:09.615801+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:09.615801+0000","last_undegraded":"2024-09-15T01:45:09.615801+0000","last_fullsized":"2024-09-15T01:45:09.615801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:24.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'62","reported_seq":309,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.928962+0000","last_change":"2024-09-15T01:45:10.644058+0000","last_active":"2024-09-15T01:45:10.928962+0000","last_peered":"2024-09-15T01:45:10.928962+0000","last_clean":"2024-09-15T01:45:10.928962+0000","last_became_active":"2024-09-15T01:44:33.551249+0000","last_became_peered":"2024-09-15T01:44:33.551249+0000","last_unstale":"2024-09-15T01:45:10.928962+0000","last_undegraded":"2024-09-15T01:45:10.928962+0000","last_fullsized":"2024-09-15T01:45:10.928962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012676863,"stat_sum":{"num_bytes":6930432,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":9755,"num_write":31,"num_write_kb":9728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,6],"acting":[0,1,4,7,8,10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.621924+0000","last_change":"2024-09-15T01:43:57.417335+0000","last_active":"2024-09-15T01:45:09.621924+0000","last_peered":"2024-09-15T01:45:09.621924+0000","last_clean":"2024-09-15T01:45:09.621924+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:09.621924+0000","last_undegraded":"2024-09-15T01:45:09.621924+0000","last_fullsized":"2024-09-15T01:45:09.621924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:16.370594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"72'34","reported_seq":274,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.641936+0000","last_change":"2024-09-15T01:45:10.642256+0000","last_active":"2024-09-15T01:45:10.641936+0000","last_peered":"2024-09-15T01:45:10.641936+0000","last_clean":"2024-09-15T01:45:10.641936+0000","last_became_active":"2024-09-15T01:44:14.534165+0000","last_became_peered":"2024-09-15T01:44:14.534165+0000","last_unstale":"2024-09-15T01:45:10.641936+0000","last_undegraded":"2024-09-15T01:45:10.641936+0000","last_fullsized":"2024-09-15T01:45:10.641936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:23.075163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029593390000000001,"stat_sum":{"num_bytes":2048000,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1255,"num_write":36,"num_write_kb":2400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0,10,9,4,1],"acting":[3,2,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.633469+0000","last_change":"2024-09-15T01:44:05.529403+0000","last_active":"2024-09-15T01:45:09.633469+0000","last_peered":"2024-09-15T01:45:09.633469+0000","last_clean":"2024-09-15T01:45:09.633469+0000","last_became_active":"2024-09-15T01:44:05.528685+0000","last_became_peered":"2024-09-15T01:44:05.528685+0000","last_unstale":"2024-09-15T01:45:09.633469+0000","last_undegraded":"2024-09-15T01:45:09.633469+0000","last_fullsized":"2024-09-15T01:45:09.633469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:20.793187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"91'69","reported_seq":376,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.931748+0000","last_change":"2024-09-15T01:45:10.630908+0000","last_active":"2024-09-15T01:45:10.931748+0000","last_peered":"2024-09-15T01:45:10.931748+0000","last_clean":"2024-09-15T01:45:10.931748+0000","last_became_active":"2024-09-15T01:44:33.548174+0000","last_became_peered":"2024-09-15T01:44:33.548174+0000","last_unstale":"2024-09-15T01:45:10.931748+0000","last_undegraded":"2024-09-15T01:45:10.931748+0000","last_fullsized":"2024-09-15T01:45:10.931748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:11.395908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071197800000000005,"stat_sum":{"num_bytes":11862016,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":41255,"num_write":65,"num_write_kb":11856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9895936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,9,3,10,2,6,0],"acting":[1,8,9,3,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.622424+0000","last_change":"2024-09-15T01:43:57.417397+0000","last_active":"2024-09-15T01:45:09.622424+0000","last_peered":"2024-09-15T01:45:09.622424+0000","last_clean":"2024-09-15T01:45:09.622424+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:09.622424+0000","last_undegraded":"2024-09-15T01:45:09.622424+0000","last_fullsized":"2024-09-15T01:45:09.622424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:14.111551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"91'188","reported_seq":475,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.921368+0000","last_change":"2024-09-15T01:45:10.647298+0000","last_active":"2024-09-15T01:45:10.921368+0000","last_peered":"2024-09-15T01:45:10.921368+0000","last_clean":"2024-09-15T01:45:10.921368+0000","last_became_active":"2024-09-15T01:44:33.551490+0000","last_became_peered":"2024-09-15T01:44:33.551490+0000","last_unstale":"2024-09-15T01:45:10.921368+0000","last_undegraded":"2024-09-15T01:45:10.921368+0000","last_fullsized":"2024-09-15T01:45:10.921368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:30.747962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016969673000000001,"stat_sum":{"num_bytes":26673152,"num_objects":11,"num_object_clones":3,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":64,"num_read_kb":38046,"num_write":144,"num_write_kb":39648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5160960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,6,7,1,8,10],"acting":[9,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.3","version":"91'37","reported_seq":316,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.887425+0000","last_change":"2024-09-15T01:45:10.646475+0000","last_active":"2024-09-15T01:45:10.887425+0000","last_peered":"2024-09-15T01:45:10.887425+0000","last_clean":"2024-09-15T01:45:10.887425+0000","last_became_active":"2024-09-15T01:44:33.554140+0000","last_became_peered":"2024-09-15T01:44:33.554140+0000","last_unstale":"2024-09-15T01:45:10.887425+0000","last_undegraded":"2024-09-15T01:45:10.887425+0000","last_fullsized":"2024-09-15T01:45:10.887425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.139848+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.022459454,"stat_sum":{"num_bytes":3981312,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":1349,"num_write":18,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2277376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,2,8,9,10],"acting":[4,3,0,6,2,8,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.2","version":"23'2","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.622151+0000","last_change":"2024-09-15T01:44:33.542100+0000","last_active":"2024-09-15T01:45:09.622151+0000","last_peered":"2024-09-15T01:45:09.622151+0000","last_clean":"2024-09-15T01:45:09.622151+0000","last_became_active":"2024-09-15T01:44:33.541651+0000","last_became_peered":"2024-09-15T01:44:33.541651+0000","last_unstale":"2024-09-15T01:45:09.622151+0000","last_undegraded":"2024-09-15T01:45:09.622151+0000","last_fullsized":"2024-09-15T01:45:09.622151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:43:54.365498+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:18.093385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"91'139","reported_seq":379,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.906528+0000","last_change":"2024-09-15T01:45:10.646776+0000","last_active":"2024-09-15T01:45:10.906528+0000","last_peered":"2024-09-15T01:45:10.906528+0000","last_clean":"2024-09-15T01:45:10.906528+0000","last_became_active":"2024-09-15T01:44:33.548478+0000","last_became_peered":"2024-09-15T01:44:33.548478+0000","last_unstale":"2024-09-15T01:45:10.906528+0000","last_undegraded":"2024-09-15T01:45:10.906528+0000","last_fullsized":"2024-09-15T01:45:10.906528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:27.310629+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024370435999999999,"stat_sum":{"num_bytes":18563072,"num_objects":8,"num_object_clones":2,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":15813,"num_write":130,"num_write_kb":29104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,6,1,7,8,0,3],"acting":[4,9,6,1,7,8,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.815255+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:10.815255+0000","last_peered":"2024-09-15T01:45:10.815255+0000","last_clean":"2024-09-15T01:45:10.815255+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:10.815255+0000","last_undegraded":"2024-09-15T01:45:10.815255+0000","last_fullsized":"2024-09-15T01:45:10.815255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'80","reported_seq":383,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.642708+0000","last_change":"2024-09-15T01:45:10.643738+0000","last_active":"2024-09-15T01:45:10.642708+0000","last_peered":"2024-09-15T01:45:10.642708+0000","last_clean":"2024-09-15T01:45:10.642708+0000","last_became_active":"2024-09-15T01:44:33.551949+0000","last_became_peered":"2024-09-15T01:44:33.551949+0000","last_unstale":"2024-09-15T01:45:10.642708+0000","last_undegraded":"2024-09-15T01:45:10.642708+0000","last_fullsized":"2024-09-15T01:45:10.642708+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012249695999999999,"stat_sum":{"num_bytes":7995392,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":15662,"num_write":41,"num_write_kb":12496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":8044544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,3,8,9,6,1],"acting":[0,4,10,3,8,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T01:45:12.009728+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:12.009728+0000","last_peered":"2024-09-15T01:45:12.009728+0000","last_clean":"2024-09-15T01:45:12.009728+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:12.009728+0000","last_undegraded":"2024-09-15T01:45:12.009728+0000","last_fullsized":"2024-09-15T01:45:12.009728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'66","reported_seq":313,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.918563+0000","last_change":"2024-09-15T01:45:10.641290+0000","last_active":"2024-09-15T01:45:10.918563+0000","last_peered":"2024-09-15T01:45:10.918563+0000","last_clean":"2024-09-15T01:45:10.918563+0000","last_became_active":"2024-09-15T01:44:33.553924+0000","last_became_peered":"2024-09-15T01:44:33.553924+0000","last_unstale":"2024-09-15T01:45:10.918563+0000","last_undegraded":"2024-09-15T01:45:10.918563+0000","last_fullsized":"2024-09-15T01:45:10.918563+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:10.985856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021946779999999998,"stat_sum":{"num_bytes":2228224,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":7063,"num_write":62,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,8,1,2,10],"acting":[3,9,6,4,8,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"2.3","version":"84'2","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T01:45:09.619390+0000","last_change":"2024-09-15T01:45:03.802676+0000","last_active":"2024-09-15T01:45:09.619390+0000","last_peered":"2024-09-15T01:45:09.619390+0000","last_clean":"2024-09-15T01:45:09.619390+0000","last_became_active":"2024-09-15T01:44:30.530642+0000","last_became_peered":"2024-09-15T01:44:30.530642+0000","last_unstale":"2024-09-15T01:45:09.619390+0000","last_undegraded":"2024-09-15T01:45:09.619390+0000","last_fullsized":"2024-09-15T01:45:09.619390+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:08.522578+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":227,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T01:45:11.526804+0000","last_change":"2024-09-15T01:45:01.074698+0000","last_active":"2024-09-15T01:45:11.526804+0000","last_peered":"2024-09-15T01:45:11.526804+0000","last_clean":"2024-09-15T01:45:11.526804+0000","last_became_active":"2024-09-15T01:44:14.520194+0000","last_became_peered":"2024-09-15T01:44:14.520194+0000","last_unstale":"2024-09-15T01:45:11.526804+0000","last_undegraded":"2024-09-15T01:45:11.526804+0000","last_fullsized":"2024-09-15T01:45:11.526804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:18.683034+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"83'29","reported_seq":292,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.630434+0000","last_change":"2024-09-15T01:45:10.630897+0000","last_active":"2024-09-15T01:45:10.630434+0000","last_peered":"2024-09-15T01:45:10.630434+0000","last_clean":"2024-09-15T01:45:10.630434+0000","last_became_active":"2024-09-15T01:44:33.555455+0000","last_became_peered":"2024-09-15T01:44:33.555455+0000","last_unstale":"2024-09-15T01:45:10.630434+0000","last_undegraded":"2024-09-15T01:45:10.630434+0000","last_fullsized":"2024-09-15T01:45:10.630434+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:12.770472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010078909999999999,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":5814,"num_write":16,"num_write_kb":4976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,2,4,3,9,10,7],"acting":[8,1,2,4,3,9,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.9","version":"79'44","reported_seq":264,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.638944+0000","last_change":"2024-09-15T01:45:10.641844+0000","last_active":"2024-09-15T01:45:10.638944+0000","last_peered":"2024-09-15T01:45:10.638944+0000","last_clean":"2024-09-15T01:45:10.638944+0000","last_became_active":"2024-09-15T01:44:33.554096+0000","last_became_peered":"2024-09-15T01:44:33.554096+0000","last_unstale":"2024-09-15T01:45:10.638944+0000","last_undegraded":"2024-09-15T01:45:10.638944+0000","last_fullsized":"2024-09-15T01:45:10.638944+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:22.584117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003034321,"stat_sum":{"num_bytes":4325376,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":18,"num_write_kb":5456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1261568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7,9,0,1,4,10],"acting":[8,3,7,9,0,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.a","version":"84'22","reported_seq":212,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.620833+0000","last_change":"2024-09-15T01:45:10.622282+0000","last_active":"2024-09-15T01:45:10.620833+0000","last_peered":"2024-09-15T01:45:10.620833+0000","last_clean":"2024-09-15T01:45:10.620833+0000","last_became_active":"2024-09-15T01:44:46.539838+0000","last_became_peered":"2024-09-15T01:44:46.539838+0000","last_unstale":"2024-09-15T01:45:10.620833+0000","last_undegraded":"2024-09-15T01:45:10.620833+0000","last_fullsized":"2024-09-15T01:45:10.620833+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018409209999999999,"stat_sum":{"num_bytes":2899968,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":9063,"num_write":28,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,10,8,9,3,7],"acting":[2,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.b","version":"83'49","reported_seq":294,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.641489+0000","last_change":"2024-09-15T01:45:10.641834+0000","last_active":"2024-09-15T01:45:10.641489+0000","last_peered":"2024-09-15T01:45:10.641489+0000","last_clean":"2024-09-15T01:45:10.641489+0000","last_became_active":"2024-09-15T01:44:14.534602+0000","last_became_peered":"2024-09-15T01:44:14.534602+0000","last_unstale":"2024-09-15T01:45:10.641489+0000","last_undegraded":"2024-09-15T01:45:10.641489+0000","last_fullsized":"2024-09-15T01:45:10.641489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:17.646194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026007080000000002,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":23484,"num_write":52,"num_write_kb":8208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,9,7,1,6],"acting":[3,0,4,8,9,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.c","version":"91'66","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.926377+0000","last_change":"2024-09-15T01:45:10.626199+0000","last_active":"2024-09-15T01:45:10.926377+0000","last_peered":"2024-09-15T01:45:10.926377+0000","last_clean":"2024-09-15T01:45:10.926377+0000","last_became_active":"2024-09-15T01:44:33.554020+0000","last_became_peered":"2024-09-15T01:44:33.554020+0000","last_unstale":"2024-09-15T01:45:10.926377+0000","last_undegraded":"2024-09-15T01:45:10.926377+0000","last_fullsized":"2024-09-15T01:45:10.926377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:15.891059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021739039999999999,"stat_sum":{"num_bytes":4177920,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":12533,"num_write":67,"num_write_kb":12080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,10,9,2,0,7],"acting":[4,3,6,10,9,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.d","version":"91'44","reported_seq":334,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T01:45:12.538111+0000","last_change":"2024-09-15T01:45:12.538449+0000","last_active":"2024-09-15T01:45:12.538111+0000","last_peered":"2024-09-15T01:45:12.538111+0000","last_clean":"2024-09-15T01:45:12.538111+0000","last_became_active":"2024-09-15T01:44:33.552142+0000","last_became_peered":"2024-09-15T01:44:33.552142+0000","last_unstale":"2024-09-15T01:45:12.538111+0000","last_undegraded":"2024-09-15T01:45:12.538111+0000","last_fullsized":"2024-09-15T01:45:12.538111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:34.473466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049082900000000003,"stat_sum":{"num_bytes":3014656,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":11733,"num_write":43,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3080192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,9,6,1,3,4,0],"acting":[7,10,9,6,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"34","length":"3"},{"start":"3c","length":"1"}]},{"pgid":"3.e","version":"91'57","reported_seq":281,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T01:45:10.622304+0000","last_change":"2024-09-15T01:45:10.622304+0000","last_active":"2024-09-15T01:45:10.622304+0000","last_peered":"2024-09-15T01:45:10.622304+0000","last_clean":"2024-09-15T01:45:10.622304+0000","last_became_active":"2024-09-15T01:44:14.525437+0000","last_became_peered":"2024-09-15T01:44:14.525437+0000","last_unstale":"2024-09-15T01:45:10.622304+0000","last_undegraded":"2024-09-15T01:45:10.622304+0000","last_fullsized":"2024-09-15T01:45:10.622304+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:25.758952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741352,"stat_sum":{"num_bytes":12615680,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":23,"num_read_kb":29433,"num_write":32,"num_write_kb":14800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1,4,0,9,3,10],"acting":[2,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"30","length":"2"},{"start":"34","length":"3"}]},{"pgid":"3.f","version":"89'36","reported_seq":305,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T01:45:12.535836+0000","last_change":"2024-09-15T01:45:12.537986+0000","last_active":"2024-09-15T01:45:12.535836+0000","last_peered":"2024-09-15T01:45:12.535836+0000","last_clean":"2024-09-15T01:45:12.535836+0000","last_became_active":"2024-09-15T01:44:14.525529+0000","last_became_peered":"2024-09-15T01:44:14.525529+0000","last_unstale":"2024-09-15T01:45:12.535836+0000","last_undegraded":"2024-09-15T01:45:12.535836+0000","last_fullsized":"2024-09-15T01:45:12.535836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:44:06.521343+0000","last_clean_scrub_stamp":"2024-09-15T01:44:06.521343+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:35.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023068070000000001,"stat_sum":{"num_bytes":4653056,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":31502,"num_write":27,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,8,2,9,10,0,4],"acting":[7,1,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"34","length":"3"},{"start":"3c","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T01:45:12.514929+0000","last_change":"2024-09-15T01:45:12.514929+0000","last_active":"2024-09-15T01:45:12.514929+0000","last_peered":"2024-09-15T01:45:12.514929+0000","last_clean":"2024-09-15T01:45:12.514929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T01:45:12.514929+0000","last_undegraded":"2024-09-15T01:45:12.514929+0000","last_fullsized":"2024-09-15T01:45:12.514929+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:12.514929+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.514929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":118636544,"num_objects":62,"num_object_clones":14,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":2,"num_read":441,"num_read_kb":253762,"num_write":810,"num_write_kb":192928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":39174144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":242868224,"data_stored":295051264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1022,"ondisk_log_size":1022,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411348,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321752,"kb_used_data":20236,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050088,"statfs":{"total":96636764160,"available":96307290112,"internally_reserved":0,"allocated":20721664,"data_stored":25357830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.194}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.052}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.363}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.4139999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.903}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":2.04},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.6419999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.9870000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.871}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.6859999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.522},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.821}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]}]},{"osd":11,"up_from":0,"seq":73014444050,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":10,"up_from":17,"seq":73014444051,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":324516,"kb_used_data":23000,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94047324,"statfs":{"total":96636764160,"available":96304459776,"internally_reserved":0,"allocated":23552000,"data_stored":28821836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.753},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0109999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.234}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.579}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4330000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.4570000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.383},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.536}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.677},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.702}]}]},{"osd":9,"up_from":17,"seq":73014444050,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330420,"kb_used_data":28904,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94041420,"statfs":{"total":96636764160,"available":96298414080,"internally_reserved":0,"allocated":29597696,"data_stored":31668556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.153},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.93000000000000005}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.2789999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.1220000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4219999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.48},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.452},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.3560000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.569},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5449999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328708,"kb_used_data":27184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043132,"statfs":{"total":96636764160,"available":96300167168,"internally_reserved":0,"allocated":27836416,"data_stored":31086637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.518}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7030000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.347}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.4319999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.49},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7470000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.782}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.379},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.839}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.1890000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.268}]}]},{"osd":6,"up_from":17,"seq":73014444051,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":324328,"kb_used_data":22812,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94047512,"statfs":{"total":96636764160,"available":96304652288,"internally_reserved":0,"allocated":23359488,"data_stored":25422719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.86299999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.6419999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.232}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.0680000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.276}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.6859999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.879},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":2.0209999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.839},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.556}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.9650000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":2.0739999999999998}]}]},{"osd":5,"up_from":0,"seq":73014444051,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":17,"seq":73014444050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322128,"kb_used_data":20612,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94049712,"statfs":{"total":96636764160,"available":96306905088,"internally_reserved":0,"allocated":21106688,"data_stored":24131916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.004},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3939999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.4750000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.105},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.79800000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.532},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.167}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.21},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.274},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.9890000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":2.0430000000000001},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.675}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":2.093}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329180,"kb_used_data":27656,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94042660,"statfs":{"total":96636764160,"available":96299683840,"internally_reserved":0,"allocated":28319744,"data_stored":31729709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.86599999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.216},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.181},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.083}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.054},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.472}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.2010000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.397},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.1240000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.171},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.5669999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.367},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.2769999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6240000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444051,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319044,"kb_used_data":17528,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052796,"statfs":{"total":96636764160,"available":96310063104,"internally_reserved":0,"allocated":17948672,"data_stored":18553164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.97999999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":1.052},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.74099999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.93200000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.091},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":1.49}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.006},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":1.179}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":1.244},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.5569999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.7569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.843},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.3540000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444050,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328552,"kb_used_data":27036,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043288,"statfs":{"total":96636764160,"available":96300326912,"internally_reserved":0,"allocated":27684864,"data_stored":33016159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.80200000000000005}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98899999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0649999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.359},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.607}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.4410000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.54}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":1.825},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.4059999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.518},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.798}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.3220000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325056,"kb_used_data":23532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046784,"statfs":{"total":96636764160,"available":96303906816,"internally_reserved":0,"allocated":24096768,"data_stored":26932345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.97899999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.018},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.755}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.90800000000000003}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.137}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.2210000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.248},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.0609999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.2909999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21569536,"data_stored":25817088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26599424,"data_stored":32350208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16904192,"data_stored":20287488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25743360,"data_stored":30752768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20025344,"data_stored":23465984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8859648,"data_stored":9601024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21864448,"data_stored":28151808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":19083264,"data_stored":24145920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":26329088,"data_stored":32616448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28639232,"data_stored":34926592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":22470656,"data_stored":28155904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:16.477 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-15T01:45:16.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-15T01:45:16.478 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:16.463+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:16.570 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:16.567+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:16.609 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:16.610 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:16.610 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-15T01:45:16.610 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T01:45:16.652 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.656 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.673 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.675 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.676 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.677 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.647+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.681 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.682 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.684 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.685 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.685 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-09-15T01:45:16.667+0000 7f17ca38b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22078) UID: 0 2024-09-15T01:45:16.685 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.686 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is set 2024-09-15T01:45:16.687 INFO:teuthology.orchestra.run.smithi033.stderr:clear 3.a pg_upmap_items mapping 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.688 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.689 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.659+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.703 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.706 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3ba7d5700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.711 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.711 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.711 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.712 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.712 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.713 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.714 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.714 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.715 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.718 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.718 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.720 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.721 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.721 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.721 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.721 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.722 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.658+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.723 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.724 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.662+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.725 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.726 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.727 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.666+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.728 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.729 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.730 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.731 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.732 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.733 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.734 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.670+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.735 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.674+0000 7f69c7366700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.736 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.678+0000 7f69cb36e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.663+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.746 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.667+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.748 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.671+0000 7fa15758a700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.679+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.679+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.755 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.755 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.755 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.675+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c988f700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.758 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.683+0000 7fe4c5887700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.781 INFO:tasks.ceph.osd.9.smithi155.stderr:2024-09-15T01:45:16.770+0000 7f69e4e21700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22351) UID: 0 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.787 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.788 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.651+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.789 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.790 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.791 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.792 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.793 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.794 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.795 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.655+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.796 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.797 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.798 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.799 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.800 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.659+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.801 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.802 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.803 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.804 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.805 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.806 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.807 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.808 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.809 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.810 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.663+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.811 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.812 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.813 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4ad9e9700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.814 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.815 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.816 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.817 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.818 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.819 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.820 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.821 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.822 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.823 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.823 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.823 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.823 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.823 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.824 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.825 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.826 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.827 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.667+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.829 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.671+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.719+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.830 INFO:tasks.ceph.osd.4.smithi047.stderr:2024-09-15T01:45:16.759+0000 7fc4b19f1700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.842 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.843 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.844 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.848 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.850 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa15b592700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.687+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.854 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.691+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.856 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.861 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.862 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.695+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.875 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.876 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.699+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.880 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.703+0000 7fa15958e700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.799+0000 7fa15d596700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.882 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:16.882 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-15T01:45:16.882 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-15T01:45:16.883 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-15T01:45:16.883 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:16.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T01:45:16.883 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.827+0000 7fa15cd95700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.884 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-09-15T01:45:16.859+0000 7fb3b67cd700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.884 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.863+0000 7fa158d8d700 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,2,3,4,5,6,7 2024-09-15T01:45:16.886 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-15T01:45:16.879+0000 7fe4e6348700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22080) UID: 0 2024-09-15T01:45:16.986 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:16.979+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:205 3.2s0 3:4ddcdeb3:::smithi03327189-33:2 [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 65.489050192999997, 2024-09-15T01:45:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011672814, 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 205 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509472+0000", 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7800000000000002e-07 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509475+0000", 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9040000000000001e-06 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509482+0000", 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4500000000000001e-06 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.509486+0000", 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0470000000000004e-06 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514158+0000", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046723900000000002 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.514269+0000", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111014 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:11.521145+0000", 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068755309999999998 2024-09-15T01:45:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:296 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.480075292999999, 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015808717, 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 296 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518447+0000", 2024-09-15T01:45:17.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518449+0000", 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5459999999999998e-06 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518461+0000", 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2204999999999999e-05 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518490+0000", 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9034999999999999e-05 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518498+0000", 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6299999999999998e-06 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518584+0000", 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6314000000000001e-05 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518597+0000", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2327000000000001e-05 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523155+0000", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045585749999999996 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.523516+0000", 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036050900000000001 2024-09-15T01:45:17.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.534255+0000", 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010739576000000001 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:297 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.480007630999999, 2024-09-15T01:45:17.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017510628, 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 297 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518514+0000", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518515+0000", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2600000000000002e-07 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518519+0000", 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-15T01:45:17.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518526+0000", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5869999999999998e-06 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518535+0000", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518677+0000", 2024-09-15T01:45:17.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014221000000000001 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518686+0000", 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7139999999999999e-06 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527435+0000", 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087497770000000002 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527564+0000", 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012872200000000001 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536025+0000", 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084608670000000004 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:298 3.7s0 3:e5da12c9:::smithi03327189-25:d [read 0~409600,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-15T01:45:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.479981889999998, 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017764094000000001, 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518540+0000", 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518541+0000", 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7300000000000001e-07 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518544+0000", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2160000000000002e-06 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518550+0000", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518554+0000", 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.625e-06 2024-09-15T01:45:17.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518715+0000", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000161466 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.518724+0000", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4330000000000004e-06 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.527979+0000", 2024-09-15T01:45:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0092554899999999999 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.528065+0000", 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6386999999999998e-05 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:22.536304+0000", 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0082386500000000001 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:536 3.bs0 3:dc585df2:::smithi03327189-7:head [copy-from ver 29] snapc 19=[18] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 39.374711966, 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016816190000000002, 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 536 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623810+0000", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623812+0000", 2024-09-15T01:45:17.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3080000000000002e-06 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623822+0000", 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623843+0000", 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0934e-05 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623853+0000", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0040999999999999e-05 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.623973+0000", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012021800000000001 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.624131+0000", 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015810600000000001 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.633116+0000", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0089850139999999995 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640086+0000", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069697969999999998 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640524+0000", 2024-09-15T01:45:17.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043788600000000002 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:37.640626+0000", 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102507 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:641 3.2s0 3:47035a48:::smithi03327189-20:head [copy-from ver 26] snapc 1f=[1f,1c] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.395403354999999, 2024-09-15T01:45:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037649145000000002, 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 641 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603119+0000", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603123+0000", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.036e-06 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603142+0000", 2024-09-15T01:45:17.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9879999999999999e-05 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603195+0000", 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2873e-05 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603217+0000", 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1296000000000001e-05 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603353+0000", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013684299999999999 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603377+0000", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.336e-05 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610830+0000", 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074532460000000002 2024-09-15T01:45:17.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.610975+0000", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014536700000000001 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.631177+0000", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020201798 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.636731+0000", 2024-09-15T01:45:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055533889999999997 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640677+0000", 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039463969999999999 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.640768+0000", 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0660000000000003e-05 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:642 3.2s0 3:47035a48:::smithi03327189-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 34.395269890000002, 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031465280999999998, 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 642 2024-09-15T01:45:17.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603252+0000", 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603254+0000", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0640000000000001e-06 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603261+0000", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7830000000000001e-06 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603282+0000", 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1347000000000001e-05 2024-09-15T01:45:17.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603292+0000", 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7429999999999997e-06 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603537+0000", 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000244818 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:17.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.603553+0000", 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6708999999999999e-05 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611254+0000", 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077003769999999996 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.611327+0000", 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3003999999999994e-05 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634306+0000", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022979216 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634499+0000", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019303899999999999 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:42.634717+0000", 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021818099999999999 2024-09-15T01:45:17.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=77, reqid=osd.9.0:77, at_version=72'117, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.474578610000002, 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014114875000000001, 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523943+0000", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523944+0000", 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.03e-07 2024-09-15T01:45:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523968+0000", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4196999999999999e-05 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523983+0000", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5373e-05 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.523991+0000", 2024-09-15T01:45:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.61e-06 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524050+0000", 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9197999999999998e-05 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.524062+0000", 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1790000000000001e-05 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538019+0000", 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013957405000000001 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538058+0000", 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8998999999999997e-05 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.5s1 72/53 ECSubWrite(tid=78, reqid=osd.9.0:78, at_version=72'119, trim_to=0'0, roll_forward_to=68'115))", 2024-09-15T01:45:17.047 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.470947178999999, 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011397544000000001, 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527575+0000", 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5099999999999999e-07 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527597+0000", 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1889e-05 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527609+0000", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2089000000000001e-05 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527617+0000", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1969999999999999e-06 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530509+0000", 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028926659999999999 2024-09-15T01:45:17.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530518+0000", 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2470000000000008e-06 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538951+0000", 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0084332639999999993 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.538972+0000", 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1441e-05 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 72/53 ECSubWrite(tid=180, reqid=osd.7.0:180, at_version=72'18, trim_to=0'0, roll_forward_to=70'16))", 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.470873504, 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014671546000000001, 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.052 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527648+0000", 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527649+0000", 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9299999999999998e-07 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527673+0000", 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4335e-05 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527684+0000", 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0302999999999999e-05 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.527689+0000", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530440+0000", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0027509190000000001 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.530449+0000", 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8440000000000004e-06 2024-09-15T01:45:17.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542297+0000", 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011848170999999999 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:51.542320+0000", 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620999999999999e-05 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 73/66 ECSubWrite(tid=59, reqid=osd.2.0:59, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.471783265999999, 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011892049999999999, 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526739+0000", 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5900000000000002e-07 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526791+0000", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1916999999999998e-05 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526801+0000", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7759999999999992e-06 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.526805+0000", 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2570000000000001e-06 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527105+0000", 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029965900000000003 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.527122+0000", 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6993000000000001e-05 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538613+0000", 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011490879000000001 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.538631+0000", 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8110000000000001e-05 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/53 ECSubWrite(tid=185, reqid=osd.7.0:185, at_version=73'24, trim_to=0'0, roll_forward_to=72'18))", 2024-09-15T01:45:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 24.470150270000001, 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.016700869, 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528372+0000", 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2e-07 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528400+0000", 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7503000000000001e-05 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528410+0000", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0137e-05 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.528415+0000", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4240000000000001e-06 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535553+0000", 2024-09-15T01:45:17.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0071381459999999997 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.535559+0000", 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5210000000000002e-06 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545040+0000", 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0094807480000000006 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:52.545073+0000", 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969999999999998e-05 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.ds5 76/53 ECSubWrite(tid=189, reqid=osd.7.0:189, at_version=76'28, trim_to=0'0, roll_forward_to=75'26))", 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.469783981999999, 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013688799999999999, 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528738+0000", 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2600000000000001e-07 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528759+0000", 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1240999999999999e-05 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528776+0000", 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6166000000000001e-05 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528783+0000", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5329999999999997e-06 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528831+0000", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7750000000000002e-05 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.528838+0000", 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0230000000000004e-06 2024-09-15T01:45:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542366+0000", 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013528014999999999 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.542427+0000", 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0846e-05 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 76/53 ECSubWrite(tid=145, reqid=osd.4.0:145, at_version=76'94, trim_to=0'0, roll_forward_to=75'92))", 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.467282812000001, 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012260155, 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531239+0000", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1199999999999999e-07 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531266+0000", 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019e-05 2024-09-15T01:45:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531281+0000", 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4214e-05 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.531290+0000", 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9290000000000004e-06 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532672+0000", 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013827100000000001 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.532682+0000", 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7960000000000007e-06 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543048+0000", 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010366175 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.543499+0000", 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045100000000000001 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.as6 76/66 ECSubWrite(tid=67, reqid=osd.2.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=76'21))", 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.463741415000001, 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013434306999999999, 2024-09-15T01:45:17.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534781+0000", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3700000000000001e-07 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534790+0000", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8699999999999998e-06 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534798+0000", 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3969999999999994e-06 2024-09-15T01:45:17.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.534804+0000", 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7450000000000001e-06 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535012+0000", 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020819 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.535021+0000", 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.473e-06 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548169+0000", 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013148529000000001 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:44:55.548215+0000", 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5666e-05 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:807 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.394977615999998, 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012342736999999999, 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 807 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603544+0000", 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603547+0000", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9380000000000001e-06 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603562+0000", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4431e-05 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603604+0000", 2024-09-15T01:45:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2518999999999997e-05 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603622+0000", 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7706e-05 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603719+0000", 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7546000000000004e-05 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603728+0000", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9549999999999998e-06 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608524+0000", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047951790000000001 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.608636+0000", 2024-09-15T01:45:17.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011230600000000001 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.609358+0000", 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072179799999999995 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613504+0000", 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041461270000000003 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615819+0000", 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023155229999999999 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615887+0000", 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7708999999999998e-05 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:808 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [rollback 30] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.394846806, 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012866819999999999, 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 808 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603675+0000", 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603677+0000", 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5859999999999999e-06 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603684+0000", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3220000000000004e-06 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603705+0000", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1056999999999998e-05 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603716+0000", 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1073999999999999e-05 2024-09-15T01:45:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603799+0000", 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3174e-05 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603803+0000", 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.422e-06 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610165+0000", 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006361841 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.610363+0000", 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019824 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.611276+0000", 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00091360300000000003 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613506+0000", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022295459999999998 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616474+0000", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029682599999999999 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.616542+0000", 2024-09-15T01:45:17.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7694999999999995e-05 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4632.0:809 3.2s0 3:4ddcdeb3:::smithi03327189-33:head [append 0~0] snapc 31=[31,30] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 16.394736836, 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015369404999999999, 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4632", 2024-09-15T01:45:17.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 809 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603785+0000", 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603787+0000", 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4169999999999999e-06 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603793+0000", 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.951e-06 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603822+0000", 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8274000000000001e-05 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603833+0000", 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1243000000000001e-05 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603863+0000", 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0425000000000001e-05 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.603867+0000", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2030000000000001e-06 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612025+0000", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0081579340000000004 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.612192+0000", 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016787099999999999 2024-09-15T01:45:17.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.613063+0000", 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00087049600000000001 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.615910+0000", 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0028468719999999999 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-15T01:45:17.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619073+0000", 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031635119999999998 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:00.619154+0000", 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1206999999999997e-05 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=159, reqid=client.4632.0:823, at_version=83'106, trim_to=0'0, roll_forward_to=80'103))", 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.447200001000001, 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011255777999999999, 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551322+0000", 2024-09-15T01:45:17.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551323+0000", 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2160000000000001e-06 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551804+0000", 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048123799999999999 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551819+0000", 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4656e-05 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551827+0000", 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6499999999999996e-06 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551880+0000", 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3767999999999997e-05 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.551943+0000", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2779000000000003e-05 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562559+0000", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010615820999999999 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.562578+0000", 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8649999999999999e-05 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.103 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "MOSDECSubOpWrite(3.0s7 83/53 ECSubWrite(tid=169, reqid=client.4632.0:845, at_version=83'112, trim_to=0'0, roll_forward_to=83'110))", 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.364416573, 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012514281, 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-15T01:45:17.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634105+0000", 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634107+0000", 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.31e-06 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634298+0000", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019108600000000001 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634338+0000", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300999999999999e-05 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634352+0000", 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3866000000000001e-05 2024-09-15T01:45:17.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634422+0000", 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0359999999999997e-05 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.634432+0000", 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5310000000000007e-06 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646573+0000", 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012141322 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-15T01:45:02.646620+0000", 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6505000000000003e-05 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-15T01:45:17.108 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:17.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T01:45:17.110 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-09-15T01:45:17.103+0000 7fa17804b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22074) UID: 0 2024-09-15T01:45:17.211 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T01:45:17.212 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:17.207+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:17.229 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "num_ops": 0 2024-09-15T01:45:17.230 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:17.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T01:45:17.312 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:17.307+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:17.340 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:17.340 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [], 2024-09-15T01:45:17.340 INFO:teuthology.orchestra.run.smithi047.stdout: "complaint_time": 30, 2024-09-15T01:45:17.340 INFO:teuthology.orchestra.run.smithi047.stdout: "num_blocked_ops": 0 2024-09-15T01:45:17.340 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:17.349 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T01:45:17.413 INFO:tasks.ceph.osd.5.smithi047.stderr:2024-09-15T01:45:17.411+0000 7f4febdbd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22093) UID: 0 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 20, 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 600, 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "ops": [ 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:225 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 64.934258940000007, 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020156488, 2024-09-15T01:45:17.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 225 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518652+0000", 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518654+0000", 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0360000000000001e-06 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.460 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518665+0000", 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0665e-05 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518681+0000", 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6065e-05 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.461 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518689+0000", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.0749999999999998e-06 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518786+0000", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.6317000000000006e-05 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518789+0000", 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5659999999999998e-06 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.530853+0000", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012063864000000001 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531327+0000", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000474363 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.538809+0000", 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0074815369999999999 2024-09-15T01:45:17.463 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:226 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 64.934200594000004, 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021921553, 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.464 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 226 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518711+0000", 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518712+0000", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1699999999999997e-07 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518717+0000", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6019999999999999e-06 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518726+0000", 2024-09-15T01:45:17.466 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.4950000000000008e-06 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518731+0000", 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2619999999999998e-06 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518821+0000", 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9987000000000004e-05 2024-09-15T01:45:17.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518822+0000", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.2699999999999999e-07 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.531850+0000", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013028430000000001 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532058+0000", 2024-09-15T01:45:17.468 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020773300000000001 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.540632+0000", 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0085745000000000005 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:227 3.3s0 3:c5b7678e:::smithi03327189-28:head [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:17.469 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 64.934121837000006, 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023826116000000001, 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 227 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518790+0000", 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518791+0000", 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.015e-06 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518796+0000", 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.2469999999999999e-06 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.472 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518807+0000", 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1554e-05 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518814+0000", 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.3330000000000002e-06 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518892+0000", 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8313999999999996e-05 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.473 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for peered", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.518896+0000", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.709e-06 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532501+0000", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.013605079000000001 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.532850+0000", 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00034935899999999999 2024-09-15T01:45:17.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:12.542616+0000", 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0097655060000000002 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:674 3.cs0 3:3584cd82:::smithi03327189-23:head [copy-from ver 66] snapc 22=[21,1f] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 30.936036584, 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025802416000000002, 2024-09-15T01:45:17.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 674 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516875+0000", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516878+0000", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0630000000000002e-06 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516891+0000", 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3165e-05 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516932+0000", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1193999999999999e-05 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.516951+0000", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8595999999999999e-05 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517124+0000", 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017323800000000001 2024-09-15T01:45:17.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.517137+0000", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2564e-05 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522398+0000", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052612420000000002 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.522650+0000", 2024-09-15T01:45:17.479 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00025233500000000002 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.532345+0000", 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00969509 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.534869+0000", 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0025237689999999999 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542576+0000", 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0077074259999999999 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:46.542677+0000", 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100734 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.481 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:773 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 30] snapc 2d=[2a] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.933955056999999, 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026472887000000001, 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 773 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.482 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518956+0000", 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518960+0000", 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.1769999999999998e-06 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.518990+0000", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0382000000000002e-05 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519067+0000", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.7348000000000006e-05 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519085+0000", 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8050000000000002e-05 2024-09-15T01:45:17.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519275+0000", 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00018974899999999999 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519561+0000", 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00028586399999999999 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.485 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.539158+0000", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.019596921999999999 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544838+0000", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056797690000000003 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545308+0000", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00047005099999999998 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.486 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.545429+0000", 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121575 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:774 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e78)", 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 19.933792891, 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025029231999999998, 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.487 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 774 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519118+0000", 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519121+0000", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.2330000000000001e-06 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519129+0000", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.7700000000000007e-06 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519153+0000", 2024-09-15T01:45:17.489 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.3017e-05 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519164+0000", 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1953000000000001e-05 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519853+0000", 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068810500000000005 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.490 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.519993+0000", 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014037699999999999 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.543813+0000", 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023819931999999999 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544004+0000", 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00019107799999999999 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.491 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:44:57.544148+0000", 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014376699999999999 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:821 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~753664] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.920306664, 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.024406196000000002, 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 821 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.493 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532605+0000", 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.532607+0000", 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.1720000000000001e-06 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533760+0000", 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001152761 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533784+0000", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4366e-05 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533795+0000", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1077000000000001e-05 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533897+0000", 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000102403 2024-09-15T01:45:17.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "op must wait for map", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.533911+0000", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3076e-05 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.536711+0000", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028006160000000001 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537117+0000", 2024-09-15T01:45:17.496 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00040575600000000001 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.539897+0000", 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0027803419999999999 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552030+0000", 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.012132830000000001 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.497 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.556945+0000", 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0049147720000000004 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557011+0000", 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6025000000000006e-05 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.498 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:822 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~786432] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.915723791, 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022305604, 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 822 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.499 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537188+0000", 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.537190+0000", 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.373e-06 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.500 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538247+0000", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001056981 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538272+0000", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.4587e-05 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.538284+0000", 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1972e-05 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.501 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543940+0000", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056560489999999998 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.544202+0000", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00026208399999999999 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.547032+0000", 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0028303849999999999 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552099+0000", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.005067289 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559392+0000", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072926639999999999 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.559493+0000", 2024-09-15T01:45:17.503 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010122 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:823 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~655360] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.910722736, 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020918839000000002, 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 823 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542189+0000", 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.505 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.542192+0000", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9799999999999998e-06 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543766+0000", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0015748540000000001 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543797+0000", 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.0658999999999999e-05 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.506 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543841+0000", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.4029000000000001e-05 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548696+0000", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.004854985 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.548823+0000", 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012638100000000001 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.550883+0000", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002060596 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557375+0000", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0064918859999999997 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563026+0000", 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0056509919999999996 2024-09-15T01:45:17.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.563107+0000", 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1477000000000006e-05 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:824 3.0s0 3:08ecf469:::smithi03327189-6:head [append 0~114688] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.909050704, 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021066457, 2024-09-15T01:45:17.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 824 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543861+0000", 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.543863+0000", 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.0669999999999999e-06 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545718+0000", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001855675 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545744+0000", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.5227000000000001e-05 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545753+0000", 2024-09-15T01:45:17.512 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3239999999999992e-06 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.552985+0000", 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0072324340000000003 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.553163+0000", 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00017793400000000001 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.513 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.554394+0000", 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.001230411 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557432+0000", 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0030381689999999999 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564820+0000", 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0073879770000000004 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.514 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.564927+0000", 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00010723899999999999 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:826 3.0s0 3:08ecf469:::smithi03327189-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.907126867000001, 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020342466, 2024-09-15T01:45:17.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "reached pg", 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 826 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545785+0000", 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.7899999999999999e-07 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.517 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545787+0000", 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.6980000000000001e-06 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545791+0000", 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.1069999999999998e-06 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.545793+0000", 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.278e-06 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.518 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557250+0000", 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.011456579 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for rw locks", 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.557367+0000", 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0001172 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.519 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566116+0000", 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087490199999999997 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.566127+0000", 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1205e-05 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:855 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 161] snapc 31=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.520 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.81748777, 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.040891745, 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 855 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.521 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635424+0000", 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635425+0000", 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.189e-06 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.522 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635431+0000", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.4139999999999999e-06 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635447+0000", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.5540999999999999e-05 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635455+0000", 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8159999999999994e-06 2024-09-15T01:45:17.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635541+0000", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.6370000000000001e-05 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635650+0000", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000109366 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.663664+0000", 2024-09-15T01:45:17.524 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.028014127999999999 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.667487+0000", 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0038226110000000001 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676257+0000", 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0087697060000000004 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.525 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.676315+0000", 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.8604000000000003e-05 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:856 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 14.817102777000001, 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029427498999999999, 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.526 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 856 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635809+0000", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.9199999999999999e-07 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635814+0000", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 4.2710000000000003e-06 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635824+0000", 2024-09-15T01:45:17.528 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.9879999999999999e-06 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635830+0000", 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.0870000000000003e-06 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635890+0000", 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.9703000000000001e-05 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.529 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.635946+0000", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.6369000000000002e-05 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.664989+0000", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029042743999999999 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665110+0000", 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000121499 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:02.665236+0000", 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012594599999999999 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:901 3.3s0 3:cad9bd25:::smithi03327189-12:head [copy-from ver 49] snapc 36=[36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 11.863852528000001, 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.029140195000000001, 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 901 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.532 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589059+0000", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589062+0000", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7829999999999999e-06 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589075+0000", 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3696000000000001e-05 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589110+0000", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.4675999999999998e-05 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589124+0000", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3977e-05 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589270+0000", 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014596999999999999 2024-09-15T01:45:17.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589474+0000", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00020429199999999999 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.606938+0000", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.017463523000000002 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612856+0000", 2024-09-15T01:45:17.535 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0059182130000000003 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618075+0000", 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0052193589999999998 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.618199+0000", 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012370600000000001 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.536 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:902 3.3s0 3:cad9bd25:::smithi03327189-12:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86)", 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 11.863764296999999, 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023705106, 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 902 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.537 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589147+0000", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589148+0000", 2024-09-15T01:45:17.538 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.327e-06 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589157+0000", 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.8789999999999992e-06 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589169+0000", 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2130000000000001e-05 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.539 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589176+0000", 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.6139999999999997e-06 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589740+0000", 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00056363300000000002 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.589833+0000", 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3189000000000005e-05 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.540 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612431+0000", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.022597926000000001 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612658+0000", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00022690899999999999 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:05.612852+0000", 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000194499 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:920 3.3s0 3:c5b7678e:::smithi03327189-28:head [copy-from ver 64] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.864833599000001, 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.035195668999999999, 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 920 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.542 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588078+0000", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588081+0000", 2024-09-15T01:45:17.543 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.9160000000000001e-06 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588093+0000", 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2293000000000001e-05 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588129+0000", 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 3.5559999999999998e-05 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.544 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588143+0000", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.4185999999999999e-05 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588287+0000", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014458299999999999 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588563+0000", 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00027543100000000003 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.613913+0000", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.025350342000000001 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.616276+0000", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.002362496 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623181+0000", 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0069057119999999996 2024-09-15T01:45:17.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.623273+0000", 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.2150000000000004e-05 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:921 3.3s0 3:c5b7678e:::smithi03327189-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.864736958, 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027587708999999998, 2024-09-15T01:45:17.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 921 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588174+0000", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588176+0000", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.2699999999999999e-06 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588182+0000", 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 6.2999999999999998e-06 2024-09-15T01:45:17.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588210+0000", 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 2.7872e-05 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588223+0000", 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.3183999999999999e-05 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.550 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588905+0000", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00068206500000000004 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589055+0000", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00014992299999999999 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615540+0000", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.026484735999999998 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.551 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615634+0000", 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.3851999999999995e-05 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615762+0000", 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00012850700000000001 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:924 3.cs0 3:3cf1f308:::smithi03327189-35:head [read 0~4079616,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:17.552 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.864426011999999, 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.023930099999999999, 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 924 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588485+0000", 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588487+0000", 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.254e-06 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.554 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588494+0000", 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.1829999999999997e-06 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588512+0000", 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8343e-05 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588522+0000", 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.03e-05 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.555 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.591976+0000", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0034535500000000001 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.592256+0000", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000280115 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.612415+0000", 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.020159355 2024-09-15T01:45:17.556 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:925 3.0s0 3:0899e339:::smithi03327189-38:head [copy-from ver 49] snapc 37=[37,36,35,34] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.864367213, 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.027142459000000001, 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 925 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588544+0000", 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.558 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588546+0000", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.42e-06 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588553+0000", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 7.8800000000000008e-06 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588572+0000", 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8966000000000001e-05 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588584+0000", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.1199999999999999e-05 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588679+0000", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 9.5367999999999999e-05 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588816+0000", 2024-09-15T01:45:17.560 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000137198 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_started", 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.607033+0000", 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.018216392000000001 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "sub_op_committed", 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.611398+0000", 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0043651899999999997 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.561 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "commit_sent", 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615605+0000", 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.0042073969999999999 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.615687+0000", 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 8.1447999999999997e-05 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "description": "osd_op(client.4632.0:926 3.0s0 3:0899e339:::smithi03327189-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T01:45:17.562 INFO:teuthology.orchestra.run.smithi047.stdout: "initiated_at": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "age": 10.864294191000001, 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021973917999999999, 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type_data": { 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flag_point": "started", 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "client_info": { 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "client": "client.4632", 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "client_addr": "172.21.15.33:0/1612012493", 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tid": 926 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "events": [ 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "initiated", 2024-09-15T01:45:17.563 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "header_read", 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588617+0000", 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "throttled", 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588618+0000", 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.203e-06 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.564 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "all_read", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588624+0000", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.7930000000000003e-06 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "dispatched", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588642+0000", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.8114e-05 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "queued_for_pg", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.588653+0000", 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 1.0730000000000001e-05 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589048+0000", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.00039461299999999998 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "waiting for blocked object", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.589152+0000", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000104838 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "reached_pg", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610379+0000", 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.021226630999999999 2024-09-15T01:45:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "started", 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610491+0000", 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000111718 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "event": "done", 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "time": "2024-09-15T01:45:06.610591+0000", 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 0.000100278 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T01:45:17.568 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T01:45:17.568 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T01:45:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:45:17.519+0000 7f94477fe700 0 --2- 172.21.15.33:0/1563620082 >> v2:172.21.15.47:3300/0 conn(0x7f944813e0a0 0x7f944813e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:45:17.569 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0: {} 2024-09-15T01:45:17.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T01:45:17.570 INFO:tasks.ceph.osd.7.smithi047.stderr:2024-09-15T01:45:17.567+0000 7f785a61a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22089) UID: 0 2024-09-15T01:45:17.581 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1: {} 2024-09-15T01:45:17.603 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2: {} 2024-09-15T01:45:17.625 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3: {} 2024-09-15T01:45:17.652 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4: {} 2024-09-15T01:45:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'unique_pool_0' removed 2024-09-15T01:45:17.671 INFO:tasks.ceph.osd.8.smithi155.stderr:2024-09-15T01:45:17.666+0000 7f84cc8d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22349) UID: 0 2024-09-15T01:45:17.698 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T01:45:17.706 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T01:45:17.707 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5: {} 2024-09-15T01:45:17.719 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6: {} 2024-09-15T01:45:17.737 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:17.737 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:17.738 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7: {} 2024-09-15T01:45:17.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T01:45:17.761 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8: {} 2024-09-15T01:45:17.783 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9: {} 2024-09-15T01:45:17.804 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10: {} 2024-09-15T01:45:17.827 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11: {} 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T01:45:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T01:45:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T01:45:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T01:45:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T01:45:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T01:45:17.842 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:17.842 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:17.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T01:45:17.954 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:17.954 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:17.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T01:45:18.064 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.064 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.073 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T01:45:18.181 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.181 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T01:45:18.287 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.287 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T01:45:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T01:45:18.513 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.513 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T01:45:18.626 INFO:teuthology.orchestra.run.smithi047.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.626 INFO:teuthology.orchestra.run.smithi047.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.635 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T01:45:18.732 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.733 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.741 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T01:45:18.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T01:45:18.837 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.837 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T01:45:18.945 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:18.946 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:18.955 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T01:45:19.054 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.054 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.063 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T01:45:19.162 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.162 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.171 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T01:45:19.272 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.272 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.281 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T01:45:19.385 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.385 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T01:45:19.494 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.494 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.501 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T01:45:19.600 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.600 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T01:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:noscrub is unset 2024-09-15T01:45:19.712 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.713 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T01:45:19.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T01:45:19.820 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.820 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:19.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T01:45:19.933 INFO:teuthology.orchestra.run.smithi155.stdout:ERROR: (22) Invalid argument 2024-09-15T01:45:19.933 INFO:teuthology.orchestra.run.smithi155.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T01:45:20.721 INFO:teuthology.orchestra.run.smithi033.stderr:nodeep-scrub is unset 2024-09-15T01:45:21.762 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-15T01:45:21.763 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-15T01:45:21.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-15T01:45:22.080 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-09-15T01:45:22.075+0000 7fed2b4c9700 -1 mon.a@0(leader).osd e100 definitely_dead 0 2024-09-15T01:45:22.728 INFO:teuthology.orchestra.run.smithi033.stderr:marked in osd.5. 2024-09-15T01:45:22.743 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-15T01:45:22.743 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-15T01:45:22.743 INFO:tasks.thrashosds.thrasher:Adding osd 11 2024-09-15T01:45:22.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 11 2024-09-15T01:45:23.062 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-09-15T01:45:23.055+0000 7fed2b4c9700 -1 mon.a@0(leader).osd e101 definitely_dead 0 2024-09-15T01:45:23.734 INFO:teuthology.orchestra.run.smithi033.stderr:marked in osd.11. 2024-09-15T01:45:23.749 INFO:tasks.thrashosds.thrasher:Added osd 11 2024-09-15T01:45:23.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T01:45:23.884 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a: {} 2024-09-15T01:45:23.897 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b: {} 2024-09-15T01:45:23.911 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c: {} 2024-09-15T01:45:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:45:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:45:23.914 INFO:teuthology.orchestra.run.smithi033.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T01:45:23.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T01:45:24.737 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T01:45:24.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T01:45:25.746 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T01:45:25.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T01:45:26.531 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T01:45:26.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T01:45:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T01:45:27.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T01:45:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T01:45:28.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T01:45:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T01:45:29.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T01:45:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T01:45:30.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T01:45:31.791 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T01:45:31.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T01:45:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T01:45:32.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T01:45:33.811 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T01:45:33.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:45:34.151+0000 7f2eb37fe700 0 --2- 172.21.15.33:0/1430319980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eb413d0a0 0x7f2eb40a7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:45:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T01:45:34.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T01:45:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T01:45:35.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T01:45:36.849 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T01:45:36.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T01:45:37.853 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T01:45:37.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T01:45:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T01:45:38.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T01:45:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T01:45:39.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.8 to 1 (10000) 2024-09-15T01:45:40.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-15T01:45:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-15T01:45:41.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.9 to 1 (10000) 2024-09-15T01:45:42.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-15T01:45:43.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-15T01:45:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.10 to 1 (10000) 2024-09-15T01:45:44.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-15T01:45:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-15T01:45:45.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-15T01:45:46.938 INFO:teuthology.orchestra.run.smithi033.stderr:reweighted osd.11 to 1 (10000) 2024-09-15T01:45:46.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-15T01:45:47.941 INFO:teuthology.orchestra.run.smithi033.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-15T01:45:47.955 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T01:45:47.955 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T01:45:47.955 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T01:45:47.955 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T01:45:47.955 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T01:45:47.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:45:48.254 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:48.255 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":126,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:45:47.925743+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:45:23.716905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:02.963123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:55.269427+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.973689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.365100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.444318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:16.536253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.879813+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.901906+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.457571+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:58.658046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:00.143314+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:19.883624+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":52,"length":3},{"begin":60,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:45:48.268 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T01:45:48.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:45:48.554 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:48.554 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":126,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:45:47.925743+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:45:23.716905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:02.963123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:55.269427+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.973689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.365100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.444318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:16.536253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.879813+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.901906+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.457571+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:58.658046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:00.143314+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:19.883624+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":52,"length":3},{"begin":60,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:45:48.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T01:45:48.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T01:45:48.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T01:45:48.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T01:45:48.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T01:45:48.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T01:45:48.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T01:45:48.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T01:45:48.577 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T01:45:48.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T01:45:48.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T01:45:48.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T01:45:48.846 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:45:48.849 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:45:48.850 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:45:48.851 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:48.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:45:48.870 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:45:48.886 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:45:48.891 INFO:teuthology.orchestra.run.smithi033.stdout:73014444060 2024-09-15T01:45:48.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:45:48.894 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:45:48.906 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:48.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:45:48.912 INFO:teuthology.orchestra.run.smithi033.stdout:77309411356 2024-09-15T01:45:48.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:45:48.914 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:48.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:45:48.920 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:48.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:45:49.395 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:49.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444058 for osd.2 2024-09-15T01:45:49.464 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.4 2024-09-15T01:45:49.502 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.505 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.515 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:49.521 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.8 2024-09-15T01:45:49.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.11 2024-09-15T01:45:49.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444059 for osd.5 2024-09-15T01:45:49.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.9 2024-09-15T01:45:49.565 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.3 2024-09-15T01:45:49.582 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.592 INFO:teuthology.orchestra.run.smithi033.stdout:77309411354 2024-09-15T01:45:49.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.0 2024-09-15T01:45:49.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411356 got 77309411354 for osd.7 2024-09-15T01:45:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:49.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444058 for osd.6 2024-09-15T01:45:49.628 INFO:teuthology.orchestra.run.smithi033.stdout:73014444057 2024-09-15T01:45:49.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444057 for osd.1 2024-09-15T01:45:49.650 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:49.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444058 for osd.10 2024-09-15T01:45:50.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:45:50.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:45:50.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T01:45:50.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T01:45:50.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:45:50.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T01:45:50.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:45:50.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:45:50.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T01:45:50.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T01:45:50.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:45:50.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T01:45:50.940 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:50.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444058 for osd.2 2024-09-15T01:45:51.011 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.4 2024-09-15T01:45:51.042 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.063 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.095 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.8 2024-09-15T01:45:51.097 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.9 2024-09-15T01:45:51.141 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.173 INFO:teuthology.orchestra.run.smithi033.stdout:73014444060 2024-09-15T01:45:51.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444060 for osd.5 2024-09-15T01:45:51.187 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.206 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.215 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.11 2024-09-15T01:45:51.223 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.1 2024-09-15T01:45:51.227 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:51.227 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.227 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.3 2024-09-15T01:45:51.248 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444059 for osd.10 2024-09-15T01:45:51.249 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.257 INFO:teuthology.orchestra.run.smithi033.stdout:73014444058 2024-09-15T01:45:51.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444058 got 73014444058 for osd.0 2024-09-15T01:45:51.268 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.284 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:51.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444059 for osd.6 2024-09-15T01:45:51.298 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.299 INFO:teuthology.orchestra.run.smithi033.stdout:77309411356 2024-09-15T01:45:51.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411356 got 77309411356 for osd.7 2024-09-15T01:45:51.311 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:51.974 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:45:52.276 INFO:teuthology.orchestra.run.smithi033.stdout:73014444059 2024-09-15T01:45:52.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444059 for osd.2 2024-09-15T01:45:52.288 DEBUG:teuthology.parallel:result is None 2024-09-15T01:45:52.288 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T01:45:52.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:52.579 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:52.582 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:52.594 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T01:45:52.316836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628776,"kb_used_data":18872,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833304,"statfs":{"total":1159641169920,"available":1155925303296,"internally_reserved":0,"allocated":19324928,"data_stored":12320306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406113"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303500,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068340,"statfs":{"total":96636764160,"available":96325980160,"internally_reserved":0,"allocated":2031616,"data_stored":1453687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.1439999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.792}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.5900000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.532},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.754}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.7090000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.982},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":2.0379999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5720000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1379999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.4179999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.069},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.482}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.222},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5820000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6160000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.659}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.76}]}]},{"osd":10,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.92300000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2230000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.024},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.367}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.175},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.49}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.431}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6990000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.329}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.8660000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.88},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.044},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.151}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":2.5579999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":2.6680000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4319999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.323}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.357},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5229999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":2.7250000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.403}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.89200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.081},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.212}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.1519999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.262},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.5680000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5289999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4079999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.7709999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.357}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.085},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.137}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.1030000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":2.2589999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":2.4940000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.5569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":2.3820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.749}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1679999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.107},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.85799999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.008},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.196}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.46}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.319},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.129},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.802}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.755}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.202}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.321},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.139}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.05}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.415}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.274},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.351}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.23}]}]},{"osd":2,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.622}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.579},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.613},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.5389999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.405}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.212},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444059,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.81200000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.02},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.79700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.089}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.3149999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.145}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.746},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.597}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.532}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":2.0910000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.5},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":2.169}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.9950000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:52.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:52.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:52.899 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T01:45:52.316836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628776,"kb_used_data":18872,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833304,"statfs":{"total":1159641169920,"available":1155925303296,"internally_reserved":0,"allocated":19324928,"data_stored":12320306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406113"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303500,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068340,"statfs":{"total":96636764160,"available":96325980160,"internally_reserved":0,"allocated":2031616,"data_stored":1453687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.1439999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.792}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.5900000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.532},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.754}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.7090000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.982},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":2.0379999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5720000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1379999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.4179999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.069},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.482}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.222},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5820000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6160000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.659}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.76}]}]},{"osd":10,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.92300000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2230000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.024},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.367}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.175},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.49}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.431}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6990000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.329}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.8660000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.88},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.044},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.151}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":2.5579999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":2.6680000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4319999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.323}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.357},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5229999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":2.7250000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.403}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.89200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.081},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.212}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.1519999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.262},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.5680000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5289999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4079999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.7709999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.357}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.085},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.137}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.1030000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":2.2589999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":2.4940000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.5569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":2.3820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.749}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1679999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.107},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.85799999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.008},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.196}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.46}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.319},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.129},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.802}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.755}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.202}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.321},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.139}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.05}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.415}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.274},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.351}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.23}]}]},{"osd":2,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.622}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.579},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.613},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.5389999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.405}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.212},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444059,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.81200000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.02},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.79700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.089}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.3149999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.145}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.746},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.597}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.532}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":2.0910000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.5},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":2.169}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.9950000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:52.903 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T01:45:52.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T01:45:53.208 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T01:45:53.224 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-15T01:45:53.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:45:53.509 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:53.510 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":126,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:45:47.925743+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:45:23.716905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:02.963123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:55.269427+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.973689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.365100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.444318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:16.536253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.879813+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.901906+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.457571+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:58.658046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:00.143314+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:19.883624+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":52,"length":3},{"begin":60,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:45:53.524 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:45:53.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:53.822 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:53.837 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T01:45:52.316836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628776,"kb_used_data":18872,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833304,"statfs":{"total":1159641169920,"available":1155925303296,"internally_reserved":0,"allocated":19324928,"data_stored":12320306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406113"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303500,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068340,"statfs":{"total":96636764160,"available":96325980160,"internally_reserved":0,"allocated":2031616,"data_stored":1453687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.1439999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.792}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.5900000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.532},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.754}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.7090000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.982},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":2.0379999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5720000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1379999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.4179999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.069},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.482}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.222},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5820000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6160000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.659}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.76}]}]},{"osd":10,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.92300000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2230000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.024},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.367}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.175},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.49}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.431}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6990000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.329}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.8660000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.88},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.044},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.151}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":2.5579999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":2.6680000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4319999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.323}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.357},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5229999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":2.7250000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.403}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.89200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.081},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.212}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.1519999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.262},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.5680000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5289999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4079999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.7709999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.357}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.085},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.137}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.1030000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":2.2589999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":2.4940000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.5569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":2.3820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.749}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1679999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.107},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.85799999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.008},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.196}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.46}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.319},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.129},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.802}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.755}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.202}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.321},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.139}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.05}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.415}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.274},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.351}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.23}]}]},{"osd":2,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.622}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.579},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.613},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.5389999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.405}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.212},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444059,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.81200000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.02},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.79700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.089}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.3149999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.145}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.746},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.597}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.532}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":2.0910000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.5},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":2.169}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.9950000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:53.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:54.154 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T01:45:52.316836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628776,"kb_used_data":18872,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833304,"statfs":{"total":1159641169920,"available":1155925303296,"internally_reserved":0,"allocated":19324928,"data_stored":12320306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406113"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303500,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068340,"statfs":{"total":96636764160,"available":96325980160,"internally_reserved":0,"allocated":2031616,"data_stored":1453687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.1439999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.484},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.792}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.5900000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":2.532},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.754}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.7090000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.982},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":2.0379999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5720000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1379999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.4179999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.069},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.482}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.222},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5820000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6160000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.659}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.76}]}]},{"osd":10,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.92300000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2230000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.024},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.367}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.175},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.49}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.431}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6990000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.329}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.8660000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.88},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.044},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.151}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":2.5579999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":2.6680000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4319999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.323}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.357},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5229999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":2.7250000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.403}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.89200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.081},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.212}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.1519999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.262},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.5680000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5289999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4079999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.7709999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.357}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.085},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.137}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.1030000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":2.2589999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":2.4940000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.5569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":2.3820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.749}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1679999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.107},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.85799999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.008},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.196}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.46}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.319},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.129},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.802}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.755}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.202}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.321},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.139}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.05}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.415}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.274},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.351}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.23}]}]},{"osd":2,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.622}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.579},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.613},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.5389999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.405}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.212},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444059,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.81200000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.02},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.79700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.089}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.3149999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.145}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.746},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.597}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.532}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":2.0910000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.5},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":2.169}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.9950000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:54.157 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:45:54.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:45:54.464 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:54.466 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:45:54.478 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-15T01:45:54.317436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628800,"kb_used_data":18896,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833280,"statfs":{"total":1159641169920,"available":1155925278720,"internally_reserved":0,"allocated":19349504,"data_stored":12335506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002299"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303524,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068316,"statfs":{"total":96636764160,"available":96325955584,"internally_reserved":0,"allocated":2056192,"data_stored":1468887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.159},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.68600000000000005}]},{"osd":2,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.27}]},{"osd":3,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.012}]},{"osd":4,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.1259999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.365},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.5800000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.202},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.6779999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.7090000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":1.272,"5min":1.272,"15min":1.272},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":11,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1379999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.4179999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.069},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.482}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.222},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.5820000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.6160000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.659}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.76}]}]},{"osd":10,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.92300000000000004}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.2230000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.024},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.367}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.175},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.49}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.431}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6990000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.329}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":1.234,"5min":1.234,"15min":1.234},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.8660000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.88},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.044},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.095},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.151}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":2.5579999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":2.6680000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.4319999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.323}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.357},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5229999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":2.7250000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.403}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.89200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.081},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.212}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.1519999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.262},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1919999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.5680000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5289999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4079999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.7709999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.357}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.085},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.137}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.4750000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.04},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":2.1030000000000002}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":2.2589999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":2.4940000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.5569999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":2.3820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.749}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1679999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.107},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.85799999999999998}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.008},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.196}]},{"osd":8,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.2090000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.46}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.319},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.109,"5min":1.109,"15min":1.109},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.129},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.802}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.755}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.202}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.321},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.139}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.05}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.415}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.274},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.351}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.23}]}]},{"osd":2,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":10.679,"5min":10.679,"15min":10.679},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":10.317,"5min":10.317,"15min":10.317},"last":0.622}]},{"osd":1,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.673,"5min":9.673,"15min":9.673},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":8.9369999999999994,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.6819999999999999,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.677,"15min":1.677},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.579},{"interface":"front","average":{"1min":1.992,"5min":1.992,"15min":1.992},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":13.509,"5min":13.509,"15min":13.509},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":12.188000000000001,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.613},{"interface":"front","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":12.295,"5min":12.295,"15min":12.295},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.864000000000001,"5min":12.864000000000001,"15min":12.864000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.909,"5min":1.909,"15min":1.909},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":12.9,"5min":12.9,"15min":12.9},"last":1.5389999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.405}]},{"osd":8,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":2.048,"15min":2.048},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":9.8170000000000002,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.212},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.0369999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":9.5549999999999997,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:44:54 2024","interfaces":[{"interface":"back","average":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":8.8780000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444059,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.70799999999999996}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.81200000000000006}]},{"osd":8,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.726,"15min":1.726},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.8689999999999998,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5569999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.02},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.79700000000000004}]},{"osd":10,"last update":"Sun Sep 15 01:44:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.089}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.3149999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.145}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.746},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.597}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.532}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":2.0910000000000002}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.5},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":2.169}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.9950000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:45:54.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:45:54.772 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:45:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":126,"fsid":"d386169c-ec95-4076-a6ad-eaab3ccf750b","created":"2024-09-15T01:43:41.737992+0000","modified":"2024-09-15T01:45:47.925743+0000","last_up_change":"2024-09-15T01:43:52.350040+0000","last_in_change":"2024-09-15T01:45:23.716905+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:43:52.349089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:43:54.322526+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5bbf5b49-bace-424f-b96c-91c3a25e48e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":22087},{"type":"v1","addr":"172.21.15.33:6801","nonce":22087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":22087},{"type":"v1","addr":"172.21.15.33:6803","nonce":22087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":22087},{"type":"v1","addr":"172.21.15.33:6807","nonce":22087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":22087},{"type":"v1","addr":"172.21.15.33:6805","nonce":22087}]},"public_addr":"172.21.15.33:6801/22087","cluster_addr":"172.21.15.33:6803/22087","heartbeat_back_addr":"172.21.15.33:6807/22087","heartbeat_front_addr":"172.21.15.33:6805/22087","state":["exists","up"]},{"osd":1,"uuid":"3ce2fc91-6d32-40c7-afc3-ad8f7b6d1464","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":22086},{"type":"v1","addr":"172.21.15.33:6809","nonce":22086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":22086},{"type":"v1","addr":"172.21.15.33:6811","nonce":22086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":22086},{"type":"v1","addr":"172.21.15.33:6815","nonce":22086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":22086},{"type":"v1","addr":"172.21.15.33:6813","nonce":22086}]},"public_addr":"172.21.15.33:6809/22086","cluster_addr":"172.21.15.33:6811/22086","heartbeat_back_addr":"172.21.15.33:6815/22086","heartbeat_front_addr":"172.21.15.33:6813/22086","state":["exists","up"]},{"osd":2,"uuid":"fcc939b7-c3fb-4117-af41-9eec226f3020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":22088},{"type":"v1","addr":"172.21.15.33:6817","nonce":22088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":22088},{"type":"v1","addr":"172.21.15.33:6819","nonce":22088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":22088},{"type":"v1","addr":"172.21.15.33:6823","nonce":22088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":22088},{"type":"v1","addr":"172.21.15.33:6821","nonce":22088}]},"public_addr":"172.21.15.33:6817/22088","cluster_addr":"172.21.15.33:6819/22088","heartbeat_back_addr":"172.21.15.33:6823/22088","heartbeat_front_addr":"172.21.15.33:6821/22088","state":["exists","up"]},{"osd":3,"uuid":"57f1d668-f85b-4b0c-9093-fb5fef4b91ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":22089},{"type":"v1","addr":"172.21.15.33:6825","nonce":22089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":22089},{"type":"v1","addr":"172.21.15.33:6827","nonce":22089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":22089},{"type":"v1","addr":"172.21.15.33:6831","nonce":22089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":22089},{"type":"v1","addr":"172.21.15.33:6829","nonce":22089}]},"public_addr":"172.21.15.33:6825/22089","cluster_addr":"172.21.15.33:6827/22089","heartbeat_back_addr":"172.21.15.33:6831/22089","heartbeat_front_addr":"172.21.15.33:6829/22089","state":["exists","up"]},{"osd":4,"uuid":"a274d12c-8683-491d-8656-5af5d12164ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":22096},{"type":"v1","addr":"172.21.15.47:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":22096},{"type":"v1","addr":"172.21.15.47:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":22096},{"type":"v1","addr":"172.21.15.47:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":22096},{"type":"v1","addr":"172.21.15.47:6805","nonce":22096}]},"public_addr":"172.21.15.47:6801/22096","cluster_addr":"172.21.15.47:6803/22096","heartbeat_back_addr":"172.21.15.47:6807/22096","heartbeat_front_addr":"172.21.15.47:6805/22096","state":["exists","up"]},{"osd":5,"uuid":"f0a90a47-9b16-46f6-a2a7-afa4a6f2a298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":22097},{"type":"v1","addr":"172.21.15.47:6817","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":22097},{"type":"v1","addr":"172.21.15.47:6819","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":22097},{"type":"v1","addr":"172.21.15.47:6823","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":22097},{"type":"v1","addr":"172.21.15.47:6821","nonce":22097}]},"public_addr":"172.21.15.47:6817/22097","cluster_addr":"172.21.15.47:6819/22097","heartbeat_back_addr":"172.21.15.47:6823/22097","heartbeat_front_addr":"172.21.15.47:6821/22097","state":["exists","up"]},{"osd":6,"uuid":"194e23d1-7f52-43ee-8557-52d75a3c22dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":22108},{"type":"v1","addr":"172.21.15.47:6809","nonce":22108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":22108},{"type":"v1","addr":"172.21.15.47:6811","nonce":22108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":22108},{"type":"v1","addr":"172.21.15.47:6815","nonce":22108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":22108},{"type":"v1","addr":"172.21.15.47:6813","nonce":22108}]},"public_addr":"172.21.15.47:6809/22108","cluster_addr":"172.21.15.47:6811/22108","heartbeat_back_addr":"172.21.15.47:6815/22108","heartbeat_front_addr":"172.21.15.47:6813/22108","state":["exists","up"]},{"osd":7,"uuid":"1d1c61e7-be03-4c72-bc95-e187eb95a1b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":22109},{"type":"v1","addr":"172.21.15.47:6825","nonce":22109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":22109},{"type":"v1","addr":"172.21.15.47:6827","nonce":22109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":22109},{"type":"v1","addr":"172.21.15.47:6831","nonce":22109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":22109},{"type":"v1","addr":"172.21.15.47:6829","nonce":22109}]},"public_addr":"172.21.15.47:6825/22109","cluster_addr":"172.21.15.47:6827/22109","heartbeat_back_addr":"172.21.15.47:6831/22109","heartbeat_front_addr":"172.21.15.47:6829/22109","state":["exists","up"]},{"osd":8,"uuid":"1f23463d-2214-416b-b3ba-23132f2d84ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":22357},{"type":"v1","addr":"172.21.15.155:6803","nonce":22357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":22357},{"type":"v1","addr":"172.21.15.155:6807","nonce":22357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":22357},{"type":"v1","addr":"172.21.15.155:6815","nonce":22357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":22357},{"type":"v1","addr":"172.21.15.155:6811","nonce":22357}]},"public_addr":"172.21.15.155:6803/22357","cluster_addr":"172.21.15.155:6807/22357","heartbeat_back_addr":"172.21.15.155:6815/22357","heartbeat_front_addr":"172.21.15.155:6811/22357","state":["exists","up"]},{"osd":9,"uuid":"39226fc7-a654-41c6-9d70-5ea2f69c236b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":22359},{"type":"v1","addr":"172.21.15.155:6802","nonce":22359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":22359},{"type":"v1","addr":"172.21.15.155:6806","nonce":22359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":22359},{"type":"v1","addr":"172.21.15.155:6814","nonce":22359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":22359},{"type":"v1","addr":"172.21.15.155:6810","nonce":22359}]},"public_addr":"172.21.15.155:6802/22359","cluster_addr":"172.21.15.155:6806/22359","heartbeat_back_addr":"172.21.15.155:6814/22359","heartbeat_front_addr":"172.21.15.155:6810/22359","state":["exists","up"]},{"osd":10,"uuid":"eaafc17e-407d-43aa-8083-2c56660d7db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":22358},{"type":"v1","addr":"172.21.15.155:6817","nonce":22358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":22358},{"type":"v1","addr":"172.21.15.155:6819","nonce":22358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":22358},{"type":"v1","addr":"172.21.15.155:6823","nonce":22358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":22358},{"type":"v1","addr":"172.21.15.155:6821","nonce":22358}]},"public_addr":"172.21.15.155:6817/22358","cluster_addr":"172.21.15.155:6819/22358","heartbeat_back_addr":"172.21.15.155:6823/22358","heartbeat_front_addr":"172.21.15.155:6821/22358","state":["exists","up"]},{"osd":11,"uuid":"05fdd01d-8c78-4a61-81e7-1d9816e951a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":22360},{"type":"v1","addr":"172.21.15.155:6825","nonce":22360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":22360},{"type":"v1","addr":"172.21.15.155:6827","nonce":22360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":22360},{"type":"v1","addr":"172.21.15.155:6831","nonce":22360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":22360},{"type":"v1","addr":"172.21.15.155:6829","nonce":22360}]},"public_addr":"172.21.15.155:6825/22360","cluster_addr":"172.21.15.155:6827/22360","heartbeat_back_addr":"172.21.15.155:6831/22360","heartbeat_front_addr":"172.21.15.155:6829/22360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:02.963123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:55.269427+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.973689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:11.365100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.444318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:16.536253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:14.879813+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.901906+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:05.457571+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:44:58.658046+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:00.143314+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:45:19.883624+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":52,"length":3},{"begin":60,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:45:55.786 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T01:45:55.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:55.948 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:55.949 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:55.965 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T01:45:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T01:45:56.281 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T01:45:56.281 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:56.430 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:56.430 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:56.430 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:56.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T01:45:56.737 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T01:45:56.747 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T01:45:56.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:56.898 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:56.899 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:56.899 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:56.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T01:45:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T01:45:57.235 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T01:45:57.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:57.383 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:57.383 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:57.383 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:57.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T01:45:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T01:45:57.720 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T01:45:57.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:57.864 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:57.864 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:57.864 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:57.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T01:45:58.172 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T01:45:58.183 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T01:45:58.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:58.327 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:58.328 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:58.328 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:58.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T01:45:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T01:45:58.651 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T01:45:58.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:58.793 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:58.793 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:58.793 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:58.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T01:45:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T01:45:59.405 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T01:45:59.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:45:59.562 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:45:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:45:59.562 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:45:59.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T01:45:59.890 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T01:45:59.901 INFO:tasks.ceph:Scrubbing osd.8 2024-09-15T01:45:59.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:46:00.054 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:46:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:46:00.054 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:46:00.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-15T01:46:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 8 to deep-scrub 2024-09-15T01:46:00.372 INFO:tasks.ceph:Scrubbing osd.9 2024-09-15T01:46:00.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:46:00.523 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:46:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:46:00.523 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:46:00.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-15T01:46:00.835 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 9 to deep-scrub 2024-09-15T01:46:00.845 INFO:tasks.ceph:Scrubbing osd.10 2024-09-15T01:46:00.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:46:00.992 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:46:00.992 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:46:00.992 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:46:01.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-15T01:46:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 10 to deep-scrub 2024-09-15T01:46:01.331 INFO:tasks.ceph:Scrubbing osd.11 2024-09-15T01:46:01.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:46:01.484 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-15T01:46:01.484 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:46:01.485 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-15T01:46:01.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-15T01:46:01.797 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 11 to deep-scrub 2024-09-15T01:46:01.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:46:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-15T01:46:02.119+0000 7fc67bfff700 0 --2- 172.21.15.33:0/3550607390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc67c0b22e0 0x7fc67c153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:46:02.312 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:46:02.316 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:46:02.329 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-15T01:46:00.319228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3628800,"kb_used_data":18896,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128833280,"statfs":{"total":1159641169920,"available":1155925278720,"internally_reserved":0,"allocated":19349504,"data_stored":12335506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003570"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.694668+0000","last_change":"2024-09-15T01:45:33.474575+0000","last_active":"2024-09-15T01:45:47.694668+0000","last_peered":"2024-09-15T01:45:47.694668+0000","last_clean":"2024-09-15T01:45:47.694668+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:45:47.694668+0000","last_undegraded":"2024-09-15T01:45:47.694668+0000","last_fullsized":"2024-09-15T01:45:47.694668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:33.474355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:33.474355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:54.643222+0000","scrub_duration":0.0037271740000000002,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529132+0000","last_change":"2024-09-15T01:45:21.158864+0000","last_active":"2024-09-15T01:45:47.529132+0000","last_peered":"2024-09-15T01:45:47.529132+0000","last_clean":"2024-09-15T01:45:47.529132+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:45:47.529132+0000","last_undegraded":"2024-09-15T01:45:47.529132+0000","last_fullsized":"2024-09-15T01:45:47.529132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:21.158673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:21.158673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:46.102299+0000","scrub_duration":0.0037034339999999998,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.933179+0000","last_change":"2024-09-15T01:45:42.999263+0000","last_active":"2024-09-15T01:45:46.933179+0000","last_peered":"2024-09-15T01:45:46.933179+0000","last_clean":"2024-09-15T01:45:46.933179+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:45:46.933179+0000","last_undegraded":"2024-09-15T01:45:46.933179+0000","last_fullsized":"2024-09-15T01:45:46.933179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:26.663897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:26.663897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:36.855528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:47.529150+0000","last_change":"2024-09-15T01:45:20.192444+0000","last_active":"2024-09-15T01:45:47.529150+0000","last_peered":"2024-09-15T01:45:47.529150+0000","last_clean":"2024-09-15T01:45:47.529150+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:45:47.529150+0000","last_undegraded":"2024-09-15T01:45:47.529150+0000","last_fullsized":"2024-09-15T01:45:47.529150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:20.192182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:20.192182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:40.863674+0000","scrub_duration":0.0032585470000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":129,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929228+0000","last_change":"2024-09-15T01:45:26.528375+0000","last_active":"2024-09-15T01:45:46.929228+0000","last_peered":"2024-09-15T01:45:46.929228+0000","last_clean":"2024-09-15T01:45:46.929228+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:45:46.929228+0000","last_undegraded":"2024-09-15T01:45:46.929228+0000","last_fullsized":"2024-09-15T01:45:46.929228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-15T01:45:22.149521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:22.149521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:51.672217+0000","scrub_duration":0.0037780359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.412557+0000","last_change":"2024-09-15T01:45:10.815139+0000","last_active":"2024-09-15T01:45:48.412557+0000","last_peered":"2024-09-15T01:45:48.412557+0000","last_clean":"2024-09-15T01:45:48.412557+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:45:48.412557+0000","last_undegraded":"2024-09-15T01:45:48.412557+0000","last_fullsized":"2024-09-15T01:45:48.412557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:10.814943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:10.814943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:22.268142+0000","scrub_duration":0.0036884320000000002,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413111+0000","last_change":"2024-09-15T01:45:12.009646+0000","last_active":"2024-09-15T01:45:48.413111+0000","last_peered":"2024-09-15T01:45:48.413111+0000","last_clean":"2024-09-15T01:45:48.413111+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:45:48.413111+0000","last_undegraded":"2024-09-15T01:45:48.413111+0000","last_fullsized":"2024-09-15T01:45:48.413111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:12.009511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:54.365498+0000","last_clean_scrub_stamp":"2024-09-15T01:45:12.009511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:25.833278+0000","scrub_duration":0.003601744,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":114,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:46.929265+0000","last_change":"2024-09-15T01:45:26.528454+0000","last_active":"2024-09-15T01:45:46.929265+0000","last_peered":"2024-09-15T01:45:46.929265+0000","last_clean":"2024-09-15T01:45:46.929265+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:45:46.929265+0000","last_undegraded":"2024-09-15T01:45:46.929265+0000","last_fullsized":"2024-09-15T01:45:46.929265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:45:03.802490+0000","last_clean_scrub_stamp":"2024-09-15T01:45:03.802490+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:21.411035+0000","scrub_duration":0.0073273460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":283,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-15T01:45:48.413177+0000","last_change":"2024-09-15T01:45:27.534046+0000","last_active":"2024-09-15T01:45:48.413177+0000","last_peered":"2024-09-15T01:45:48.413177+0000","last_clean":"2024-09-15T01:45:48.413177+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:45:48.413177+0000","last_undegraded":"2024-09-15T01:45:48.413177+0000","last_fullsized":"2024-09-15T01:45:48.413177+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T01:45:01.074489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:43:52.350040+0000","last_clean_scrub_stamp":"2024-09-15T01:45:01.074489+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.164578+0000","scrub_duration":0.0041245300000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303524,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068316,"statfs":{"total":96636764160,"available":96325955584,"internally_reserved":0,"allocated":2056192,"data_stored":1468887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.05}]},{"osd":2,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0009999999999999,"15min":1.004},"min":{"1min":0.70899999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3240000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.228,"15min":1.2250000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.1160000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1120000000000001,"15min":1.1200000000000001},"min":{"1min":0.73699999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1679999999999999,"15min":1.2},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4179999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.78000000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1970000000000001,"15min":1.2030000000000001},"min":{"1min":0.73099999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6659999999999999,"5min":1.7,"15min":1.7},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.1539999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.077}]},{"osd":5,"last update":"Sun Sep 15 01:45:10 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.278}]},{"osd":6,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4530000000000001,"15min":1.4490000000000001},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.387},{"interface":"front","average":{"1min":1.353,"5min":1.341,"15min":1.339},"min":{"1min":0.96199999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.716,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.0089999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6499999999999999,"15min":1.655},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.526},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.532,"15min":1.5289999999999999},"min":{"1min":1.1160000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.1749999999999998,"5min":2.27,"15min":2.27},"last":1.5649999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.6180000000000001,"15min":1.6080000000000001},"min":{"1min":1.2529999999999999,"5min":1.071,"15min":1.071},"max":{"1min":2.2650000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.76},{"interface":"front","average":{"1min":1.629,"5min":1.577,"15min":1.5680000000000001},"min":{"1min":1.341,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":1.964,"5min":2.173,"15min":2.173},"last":1.619}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6160000000000001,"15min":1.619},"min":{"1min":1.238,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.238},{"interface":"front","average":{"1min":1.694,"5min":1.7310000000000001,"15min":1.7370000000000001},"min":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"max":{"1min":2.1000000000000001,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.7190000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]}]},{"osd":11,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299364,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072476,"statfs":{"total":96636764160,"available":96330215424,"internally_reserved":0,"allocated":2052096,"data_stored":1467939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.359}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.137},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1890000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.502},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.667}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.323},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.538},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.7629999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.827}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.9730000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.855}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.915}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":2.0209999999999999}]}]},{"osd":10,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96299999999999997,"15min":0.96499999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95199999999999996,"15min":0.94599999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.98199999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1,"15min":0.996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.631,"5min":1.643,"15min":1.643},"last":1.345}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.006,"15min":0.99099999999999999},"min":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95499999999999996,"15min":0.95699999999999996},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.88300000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.129,"15min":1.1200000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9159999999999999,"5min":1.925,"15min":1.925},"last":1.181},{"interface":"front","average":{"1min":1.135,"5min":1.196,"15min":1.206},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.583,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.749}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1759999999999999,"15min":1.1659999999999999},"min":{"1min":0.97399999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.155},{"interface":"front","average":{"1min":1.292,"5min":1.321,"15min":1.3260000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.264}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2,"15min":1.1819999999999999},"min":{"1min":0.91700000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.323},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4350000000000001,"15min":1.4330000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4279999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.2969999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3220000000000001,"15min":1.3160000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.631,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.3939999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2969999999999999,"15min":1.284},"min":{"1min":0.83199999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8440000000000001,"5min":1.871,"15min":1.871},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.429,"15min":1.417},"min":{"1min":1.0509999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.53}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4710000000000001,"15min":1.4730000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.556},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.409,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.4910000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.585},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.6240000000000001}]}]},{"osd":9,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.65800000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4510000000000001,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.107},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.70899999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.145}]},{"osd":1,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.024,"15min":1.032},"min":{"1min":0.66400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4379999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.875},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0620000000000001,"15min":1.0800000000000001},"min":{"1min":0.70299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3640000000000001,"5min":2.802,"15min":2.802},"last":0.83899999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.2689999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.0269999999999999},"min":{"1min":0.72399999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.256,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.228},{"interface":"front","average":{"1min":1.153,"5min":1.0629999999999999,"15min":1.048},"min":{"1min":0.75,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.536,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.2030000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.294},"min":{"1min":0.872,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5249999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.413},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2989999999999999,"15min":1.3029999999999999},"min":{"1min":0.89000000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.0489999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.6680000000000001,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.331,"15min":1.3100000000000001},"min":{"1min":1.1000000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.911,"5min":2.343,"15min":2.343},"last":1.3140000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.345,"15min":1.339},"min":{"1min":0.98899999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.371},{"interface":"front","average":{"1min":1.405,"5min":1.391,"15min":1.389},"min":{"1min":1.175,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6899999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.4950000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:45:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.548},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.5980000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.381,"15min":1.371},"min":{"1min":1.137,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.841,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.345},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4179999999999999,"15min":1.4099999999999999},"min":{"1min":1.1020000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.5229999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.3959999999999999,"15min":1.379},"min":{"1min":1.101,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.921,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.48,"15min":1.476},"min":{"1min":1.214,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7629999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.70599999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.093}]},{"osd":3,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.323}]},{"osd":4,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2}]},{"osd":5,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.97799999999999998}]},{"osd":6,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.6180000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.569},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.161}]},{"osd":9,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.052}]},{"osd":10,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.248},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.6659999999999999,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.2929999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.458},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5980000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444061,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.423}]},{"osd":3,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.2949999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.526}]},{"osd":5,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.4750000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.401}]},{"osd":8,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.044},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.45}]},{"osd":9,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":2.0110000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.266}]},{"osd":10,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":2.0699999999999998}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444062,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298852,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072988,"statfs":{"total":96636764160,"available":96330739712,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.032,"15min":1.026},"min":{"1min":0.73199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0069999999999999,"15min":0.98499999999999999},"min":{"1min":0.69599999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.94499999999999995}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.996,"15min":0.98999999999999999},"min":{"1min":0.72499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.042,"5min":1.0740000000000001,"15min":1.079},"min":{"1min":0.63200000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.357,"5min":1.603,"15min":1.603},"last":0.63200000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.089,"15min":1.0760000000000001},"min":{"1min":0.78700000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.552,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0569999999999999,"15min":1.056},"min":{"1min":0.70299999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.71199999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.143,"15min":1.1399999999999999},"min":{"1min":0.80400000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.6040000000000001,"5min":1.726,"15min":1.726},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.2709999999999999,"15min":1.2649999999999999},"min":{"1min":0.84399999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7509999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.1879999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2390000000000001,"15min":1.2370000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.1779999999999999,"15min":1.171},"min":{"1min":0.58699999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1439999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2350000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.855,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.4179999999999999,"15min":1.4079999999999999},"min":{"1min":0.65400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.52}]},{"osd":7,"last update":"Sun Sep 15 01:45:07 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.077},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":1.4039999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:14 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.6799999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.4750000000000001,"15min":1.4650000000000001},"min":{"1min":0.98299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.365},{"interface":"front","average":{"1min":1.6899999999999999,"5min":1.633,"15min":1.623},"min":{"1min":1.101,"5min":1.044,"15min":1.044},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.1240000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.6719999999999999,"15min":1.6639999999999999},"min":{"1min":1.1859999999999999,"5min":0.999,"15min":0.999},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.238},{"interface":"front","average":{"1min":1.8,"5min":1.6499999999999999,"15min":1.625},"min":{"1min":1.2749999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.788}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6870000000000001,"5min":1.786,"15min":1.786},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.2170000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0900000000000001,"15min":1.0920000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5660000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.429,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.204,"15min":1.224},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5149999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.302,"15min":1.3320000000000001},"min":{"1min":0.83499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4930000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.405,"15min":1.4350000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.718,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.194},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3480000000000001,"5min":1.569,"15min":1.569},"last":0.94799999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.80700000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.7709999999999999,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8979999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.1359999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.5580000000000001,"15min":1.5960000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.748,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.234},{"interface":"front","average":{"1min":1.127,"5min":1.2989999999999999,"15min":1.3280000000000001},"min":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5960000000000001,"5min":1.962,"15min":1.962},"last":1.044}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.427}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.579,"15min":1.611},"min":{"1min":1.0640000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.8109999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.262},{"interface":"front","average":{"1min":1.556,"5min":1.671,"15min":1.6899999999999999},"min":{"1min":1.1899999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.9890000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.369}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.601,"15min":1.6160000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.0430000000000001,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.292},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.774,"15min":1.802},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.149,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.1910000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.6100000000000001,"15min":1.6359999999999999},"min":{"1min":1.129,"5min":1.109,"15min":1.109},"max":{"1min":1.9099999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.7210000000000001,"15min":1.7270000000000001},"min":{"1min":1.3360000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.093,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.48}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444060,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.80100000000000005}]},{"osd":2,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.76200000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.98899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.081},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.123}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.294}]},{"osd":7,"last update":"Sun Sep 15 01:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.327},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.3879999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.548},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.5169999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.492}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.6619999999999999}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.5980000000000001}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303008,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068832,"statfs":{"total":96636764160,"available":96326483968,"internally_reserved":0,"allocated":1523712,"data_stored":939959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.2929999999999999,"15min":1.373},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":10.679,"15min":10.679},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.387},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.49,"5min":10.317,"15min":10.317},"last":0.73199999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.333,"15min":1.407},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.238,"5min":9.673,"15min":9.673},"last":0.878},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.304,"15min":1.373},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1459999999999999,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.81000000000000005}]},{"osd":3,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95599999999999996,"15min":0.96899999999999997},"min":{"1min":0.42999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.48,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.5669999999999999,"15min":1.657},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4059999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":1.2789999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.575,"15min":1.643},"min":{"1min":0.64400000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6859999999999999,"5min":12.458,"15min":12.458},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.8500000000000001,"15min":1.944},"min":{"1min":0.86599999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.768,"5min":13.509,"15min":13.509},"last":1.1859999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.603,"15min":1.6759999999999999},"min":{"1min":0.59599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.613,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.149},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.762,"15min":1.841},"min":{"1min":0.83199999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.9399999999999999,"5min":12.295,"15min":12.295},"last":1.399}]},{"osd":6,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.8129999999999999,"15min":1.885},"min":{"1min":0.88300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.452,"5min":12.864000000000001,"15min":12.864000000000001},"last":1.51},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.7989999999999999,"15min":1.8720000000000001},"min":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8080000000000001,"5min":12.9,"15min":12.9},"last":1.5329999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.613}]},{"osd":8,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.944,"15min":2.0129999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.974,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.887,"15min":1.9359999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.859,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.7410000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.4430000000000001,"15min":1.425},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.9379999999999999,"15min":2.0049999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0459999999999998,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.704}]},{"osd":10,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.921,"15min":1.9750000000000001},"min":{"1min":0.95799999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.8050000000000002,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.583},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.8480000000000001,"15min":1.8879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.6890000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.677}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444060,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1527808,"data_stored":939978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0189999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.01,"5min":0.79000000000000004,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.96099999999999997}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.03,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.29,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.76400000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.492,"5min":1.595,"15min":1.595},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0609999999999999,"15min":1.0469999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6020000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.65800000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.117,"15min":1.121},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.583,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1639999999999999,"15min":1.161},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.649,"5min":1.778,"15min":1.778},"last":1.026}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6819999999999999,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1919999999999999,"15min":1.181},"min":{"1min":0.85999999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":1.702,"15min":1.702},"last":1.26}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2889999999999999,"15min":1.28},"min":{"1min":1.0569999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.752,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.226},{"interface":"front","average":{"1min":1.304,"5min":1.302,"15min":1.302},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6899999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.006}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.294}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.7070000000000001,"15min":1.7190000000000001},"min":{"1min":0.76300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.044,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.492,"15min":1.468},"min":{"1min":0.97099999999999997,"5min":0.747,"15min":0.747},"max":{"1min":2.1150000000000002,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.4359999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.452,"15min":1.421},"min":{"1min":1.02,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.482,"5min":1.3959999999999999,"15min":1.3819999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.2030000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.486,"15min":1.462},"min":{"1min":1.004,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":1.089,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.462}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444060,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303000,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068840,"statfs":{"total":96636764160,"available":96326492160,"internally_reserved":0,"allocated":1519616,"data_stored":939011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.327},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.83399999999999996}]},{"osd":4,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.01},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.302},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0289999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.349}]},{"osd":9,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.216}]},{"osd":10,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.236},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.1100000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.274},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.387}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T01:45:33.474355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=33, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T01:45:21.158673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T01:45:26.663897+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=26, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T01:45:20.192182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T01:45:22.149521+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T01:45:10.814943+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=10, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T01:45:12.009511+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.335 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T01:45:03.802490+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.336 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T01:45:01.074489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=1, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=45, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:46:02.336 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T01:46:22.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:46:22.652 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-15T01:46:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-15T01:46:22.668 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-15T01:46:22.325279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":3629332,"kb_used_data":19460,"kb_used_omap":19,"kb_used_meta":3609708,"kb_avail":1128832748,"statfs":{"total":1159641169920,"available":1155924733952,"internally_reserved":0,"allocated":19927040,"data_stored":12695780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20448,"internal_metadata":3696341024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002269"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:16.475502+0000","last_change":"2024-09-15T01:46:00.189313+0000","last_active":"2024-09-15T01:46:16.475502+0000","last_peered":"2024-09-15T01:46:16.475502+0000","last_clean":"2024-09-15T01:46:16.475502+0000","last_became_active":"2024-09-15T01:43:55.501373+0000","last_became_peered":"2024-09-15T01:43:55.501373+0000","last_unstale":"2024-09-15T01:46:16.475502+0000","last_undegraded":"2024-09-15T01:46:16.475502+0000","last_fullsized":"2024-09-15T01:46:16.475502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:46:00.189087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:46:00.189087+0000","last_clean_scrub_stamp":"2024-09-15T01:46:00.189087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:01.395753+0000","scrub_duration":0.0039620840000000003,"objects_trimmed":0,"snaptrim_duration":0.000904505,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:17.255480+0000","last_change":"2024-09-15T01:45:58.098351+0000","last_active":"2024-09-15T01:46:17.255480+0000","last_peered":"2024-09-15T01:46:17.255480+0000","last_clean":"2024-09-15T01:46:17.255480+0000","last_became_active":"2024-09-15T01:43:55.396749+0000","last_became_peered":"2024-09-15T01:43:55.396749+0000","last_unstale":"2024-09-15T01:46:17.255480+0000","last_undegraded":"2024-09-15T01:46:17.255480+0000","last_fullsized":"2024-09-15T01:46:17.255480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:45:58.098151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:45:58.098151+0000","last_clean_scrub_stamp":"2024-09-15T01:45:58.098151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:09.226763+0000","scrub_duration":0.0031038540000000001,"objects_trimmed":0,"snaptrim_duration":0.000876479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:16.552744+0000","last_change":"2024-09-15T01:46:03.434306+0000","last_active":"2024-09-15T01:46:16.552744+0000","last_peered":"2024-09-15T01:46:16.552744+0000","last_clean":"2024-09-15T01:46:16.552744+0000","last_became_active":"2024-09-15T01:45:42.998720+0000","last_became_peered":"2024-09-15T01:45:42.998720+0000","last_unstale":"2024-09-15T01:46:16.552744+0000","last_undegraded":"2024-09-15T01:46:16.552744+0000","last_fullsized":"2024-09-15T01:46:16.552744+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:46:03.434105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:46:03.434105+0000","last_clean_scrub_stamp":"2024-09-15T01:46:03.434105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:04.547477+0000","scrub_duration":0.0059014209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:17.255515+0000","last_change":"2024-09-15T01:46:08.056490+0000","last_active":"2024-09-15T01:46:17.255515+0000","last_peered":"2024-09-15T01:46:17.255515+0000","last_clean":"2024-09-15T01:46:17.255515+0000","last_became_active":"2024-09-15T01:43:55.498910+0000","last_became_peered":"2024-09-15T01:43:55.498910+0000","last_unstale":"2024-09-15T01:46:17.255515+0000","last_undegraded":"2024-09-15T01:46:17.255515+0000","last_fullsized":"2024-09-15T01:46:17.255515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:46:08.056303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:46:08.056303+0000","last_clean_scrub_stamp":"2024-09-15T01:46:08.056303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:27.702425+0000","scrub_duration":0.0030638060000000001,"objects_trimmed":0,"snaptrim_duration":0.00088637199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"126'3","reported_seq":145,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:16.475114+0000","last_change":"2024-09-15T01:45:59.614924+0000","last_active":"2024-09-15T01:46:16.475114+0000","last_peered":"2024-09-15T01:46:16.475114+0000","last_clean":"2024-09-15T01:46:16.475114+0000","last_became_active":"2024-09-15T01:45:26.527560+0000","last_became_peered":"2024-09-15T01:45:26.527560+0000","last_unstale":"2024-09-15T01:46:16.475114+0000","last_undegraded":"2024-09-15T01:46:16.475114+0000","last_fullsized":"2024-09-15T01:46:16.475114+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'3","last_scrub_stamp":"2024-09-15T01:45:59.614705+0000","last_deep_scrub":"126'3","last_deep_scrub_stamp":"2024-09-15T01:45:59.614705+0000","last_clean_scrub_stamp":"2024-09-15T01:45:59.614705+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:28.441952+0000","scrub_duration":0.0067742840000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:16.553288+0000","last_change":"2024-09-15T01:46:05.004012+0000","last_active":"2024-09-15T01:46:16.553288+0000","last_peered":"2024-09-15T01:46:16.553288+0000","last_clean":"2024-09-15T01:46:16.553288+0000","last_became_active":"2024-09-15T01:43:55.401448+0000","last_became_peered":"2024-09-15T01:43:55.401448+0000","last_unstale":"2024-09-15T01:46:16.553288+0000","last_undegraded":"2024-09-15T01:46:16.553288+0000","last_fullsized":"2024-09-15T01:46:16.553288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:46:05.003795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:46:05.003795+0000","last_clean_scrub_stamp":"2024-09-15T01:46:05.003795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:31.672820+0000","scrub_duration":0.0033051840000000001,"objects_trimmed":0,"snaptrim_duration":0.00089675699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-15T01:46:14.031489+0000","last_change":"2024-09-15T01:46:10.071111+0000","last_active":"2024-09-15T01:46:14.031489+0000","last_peered":"2024-09-15T01:46:14.031489+0000","last_clean":"2024-09-15T01:46:14.031489+0000","last_became_active":"2024-09-15T01:43:55.501414+0000","last_became_peered":"2024-09-15T01:43:55.501414+0000","last_unstale":"2024-09-15T01:46:14.031489+0000","last_undegraded":"2024-09-15T01:46:14.031489+0000","last_fullsized":"2024-09-15T01:46:14.031489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:46:10.070884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:46:10.070884+0000","last_clean_scrub_stamp":"2024-09-15T01:46:10.070884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:31.360176+0000","scrub_duration":0.0050028169999999997,"objects_trimmed":0,"snaptrim_duration":0.00067454500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":129,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-15T01:46:16.475131+0000","last_change":"2024-09-15T01:46:00.624562+0000","last_active":"2024-09-15T01:46:16.475131+0000","last_peered":"2024-09-15T01:46:16.475131+0000","last_clean":"2024-09-15T01:46:16.475131+0000","last_became_active":"2024-09-15T01:45:26.527740+0000","last_became_peered":"2024-09-15T01:45:26.527740+0000","last_unstale":"2024-09-15T01:46:16.475131+0000","last_undegraded":"2024-09-15T01:46:16.475131+0000","last_fullsized":"2024-09-15T01:46:16.475131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'2","last_scrub_stamp":"2024-09-15T01:46:00.624345+0000","last_deep_scrub":"84'2","last_deep_scrub_stamp":"2024-09-15T01:46:00.624345+0000","last_clean_scrub_stamp":"2024-09-15T01:46:00.624345+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:10.581953+0000","scrub_duration":0.0039531380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"127'94","reported_seq":299,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-15T01:46:14.031585+0000","last_change":"2024-09-15T01:46:08.145014+0000","last_active":"2024-09-15T01:46:14.031585+0000","last_peered":"2024-09-15T01:46:14.031585+0000","last_clean":"2024-09-15T01:46:14.031585+0000","last_became_active":"2024-09-15T01:45:27.532977+0000","last_became_peered":"2024-09-15T01:45:27.532977+0000","last_unstale":"2024-09-15T01:46:14.031585+0000","last_undegraded":"2024-09-15T01:46:14.031585+0000","last_fullsized":"2024-09-15T01:46:14.031585+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'94","last_scrub_stamp":"2024-09-15T01:46:08.144791+0000","last_deep_scrub":"127'94","last_deep_scrub_stamp":"2024-09-15T01:46:08.144791+0000","last_clean_scrub_stamp":"2024-09-15T01:46:08.144791+0000","objects_scrubbed":2,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:34.073965+0000","scrub_duration":0.0091967149999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":18,"seq":77309411362,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303560,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068280,"statfs":{"total":96636764160,"available":96325918720,"internally_reserved":0,"allocated":2093056,"data_stored":1491813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.3620000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:46:16 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":0.97699999999999998,"15min":0.95999999999999996},"min":{"1min":0.70099999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4530000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.179},{"interface":"front","average":{"1min":1.077,"5min":1.024,"15min":1.0149999999999999},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.071}]},{"osd":2,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0009999999999999,"15min":1.004},"min":{"1min":0.70899999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3240000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.248,"5min":1.228,"15min":1.2250000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.321}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1120000000000001,"15min":1.1200000000000001},"min":{"1min":0.73699999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1679999999999999,"15min":1.2},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.4179999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.88500000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1970000000000001,"15min":1.2030000000000001},"min":{"1min":0.73099999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6659999999999999,"5min":1.7,"15min":1.7},"last":1.127},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.1539999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.0309999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:46:12 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.85499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.272},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4470000000000001,"15min":1.4419999999999999},"min":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":1.913,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.458}]},{"osd":6,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4530000000000001,"15min":1.4490000000000001},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.377},{"interface":"front","average":{"1min":1.353,"5min":1.341,"15min":1.339},"min":{"1min":0.96199999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.716,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.0960000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6499999999999999,"15min":1.655},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.542},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.532,"15min":1.5289999999999999},"min":{"1min":1.1160000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":2.1749999999999998,"5min":2.27,"15min":2.27},"last":1.2330000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.6180000000000001,"15min":1.6080000000000001},"min":{"1min":1.2529999999999999,"5min":1.071,"15min":1.071},"max":{"1min":2.2650000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.577,"15min":1.5680000000000001},"min":{"1min":1.341,"5min":1.1719999999999999,"15min":1.1719999999999999},"max":{"1min":1.964,"5min":2.173,"15min":2.173},"last":1.629}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6160000000000001,"15min":1.619},"min":{"1min":1.238,"5min":1.2350000000000001,"15min":1.2350000000000001},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.7310000000000001,"15min":1.7370000000000001},"min":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"max":{"1min":2.1000000000000001,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.7090000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":11,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299408,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072432,"statfs":{"total":96636764160,"available":96330170368,"internally_reserved":0,"allocated":2101248,"data_stored":1498607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.042,"15min":1.048},"min":{"1min":0.69399999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5780000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.1279999999999999,"15min":1.133},"min":{"1min":0.85199999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5069999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.4790000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.149,"15min":1.181},"min":{"1min":0.77700000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.24,"5min":1.617,"15min":1.617},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1950000000000001,"15min":1.2},"min":{"1min":0.71299999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4930000000000001,"5min":1.605,"15min":1.605},"last":1.276}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3009999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.98799999999999999,"15min":0.98199999999999998},"min":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3200000000000001,"15min":1.3149999999999999},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.2490000000000001,"15min":1.254},"min":{"1min":0.80800000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.667,"5min":1.722,"15min":1.722},"last":1.708}]},{"osd":5,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.335,"15min":1.3169999999999999},"min":{"1min":0.78100000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3089999999999999,"15min":1.288},"min":{"1min":0.97099999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.6220000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.327,"15min":1.319},"min":{"1min":0.89600000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.7210000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.4990000000000001,"15min":1.4890000000000001},"min":{"1min":1.071,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.7609999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.4970000000000001,"15min":1.4830000000000001},"min":{"1min":1.222,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4079999999999999,"15min":1.411},"min":{"1min":1.0029999999999999,"5min":0.996,"15min":0.996},"max":{"1min":1.827,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.831}]},{"osd":8,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.357,"15min":1.341},"min":{"1min":0.97199999999999998,"5min":0.874,"15min":0.874},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.22},{"interface":"front","average":{"1min":1.667,"5min":1.5860000000000001,"15min":1.5720000000000001},"min":{"1min":0.996,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":2.0339999999999998}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.492,"15min":1.4870000000000001},"min":{"1min":1.139,"5min":1.008,"15min":1.008},"max":{"1min":1.9099999999999999,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":1.2050000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.915,"5min":1.95,"15min":1.95},"last":1.3759999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.696,"5min":1.53,"15min":1.502},"min":{"1min":1.2689999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.518},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":2.0209999999999999,"5min":2.036,"15min":2.036},"last":1.7969999999999999}]}]},{"osd":10,"up_from":17,"seq":73014444066,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.96299999999999997,"15min":0.96499999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95199999999999996,"15min":0.94599999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.109}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1180000000000001,"15min":1.1279999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.748},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1,"15min":0.996},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.631,"5min":1.643,"15min":1.643},"last":1.2470000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.006,"15min":0.99099999999999999},"min":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95499999999999996,"15min":0.95699999999999996},"min":{"1min":0.70099999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.96299999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.129,"15min":1.1200000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9159999999999999,"5min":1.925,"15min":1.925},"last":1.042},{"interface":"front","average":{"1min":1.135,"5min":1.196,"15min":1.206},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.583,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.004}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1759999999999999,"15min":1.1659999999999999},"min":{"1min":0.97399999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5369999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.085},{"interface":"front","average":{"1min":1.292,"5min":1.321,"15min":1.3260000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2,"15min":1.1819999999999999},"min":{"1min":0.91700000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.177},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4350000000000001,"15min":1.4330000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.395}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.2969999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.137},{"interface":"front","average":{"1min":1.357,"5min":1.3220000000000001,"15min":1.3160000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.631,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.28}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2969999999999999,"15min":1.284},"min":{"1min":0.83199999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8440000000000001,"5min":1.871,"15min":1.871},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.429,"15min":1.417},"min":{"1min":1.0509999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.458}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4710000000000001,"15min":1.4730000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.409,"15min":1.397},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.365}]},{"osd":11,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.587,"15min":1.591},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.504},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4239999999999999}]}]},{"osd":9,"up_from":17,"seq":73014444065,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.65800000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4510000000000001,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94399999999999995,"15min":0.93999999999999995},"min":{"1min":0.70899999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.748}]},{"osd":1,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.024,"15min":1.032},"min":{"1min":0.66400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4379999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.248},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0620000000000001,"15min":1.0800000000000001},"min":{"1min":0.70299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3640000000000001,"5min":2.802,"15min":2.802},"last":0.78800000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:45:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.96799999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.0269999999999999},"min":{"1min":0.72399999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.256,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.123},{"interface":"front","average":{"1min":1.153,"5min":1.0629999999999999,"15min":1.048},"min":{"1min":0.75,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.536,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.86699999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2869999999999999,"15min":1.294},"min":{"1min":0.872,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5249999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.498},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2989999999999999,"15min":1.3029999999999999},"min":{"1min":0.89000000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.0549999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.6680000000000001,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.331,"15min":1.3100000000000001},"min":{"1min":1.1000000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.911,"5min":2.343,"15min":2.343},"last":1.373}]},{"osd":6,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.345,"15min":1.339},"min":{"1min":0.98899999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.391,"15min":1.389},"min":{"1min":1.175,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6899999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.331}]},{"osd":7,"last update":"Sun Sep 15 01:46:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.357,"15min":1.3480000000000001},"min":{"1min":0.97599999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.7989999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.4379999999999999,"15min":1.4159999999999999},"min":{"1min":1.3149999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8979999999999999,"5min":2.391,"15min":2.391},"last":1.671}]},{"osd":8,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.381,"15min":1.371},"min":{"1min":1.137,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.841,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4179999999999999,"15min":1.4099999999999999},"min":{"1min":1.1020000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.4530000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.3959999999999999,"15min":1.379},"min":{"1min":1.101,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.921,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.397},{"interface":"front","average":{"1min":1.504,"5min":1.48,"15min":1.476},"min":{"1min":1.214,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.7629999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.633}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":8,"up_from":17,"seq":73014444065,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.91100000000000003,"15min":0.89500000000000002},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4850000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85899999999999999,"15min":0.84999999999999998},"min":{"1min":0.56599999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.81200000000000006}]},{"osd":1,"last update":"Sun Sep 15 01:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.0529999999999999,"15min":1.0389999999999999},"min":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.167,"15min":1.175},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3939999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.198}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1839999999999999,"15min":1.198},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.651,"5min":1.821,"15min":1.821},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.141,"15min":1.139},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.5489999999999999,"5min":1.579,"15min":1.579},"last":1.099}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2270000000000001,"15min":1.218},"min":{"1min":0.81200000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4850000000000001,"15min":1.488},"min":{"1min":1.2,"5min":1.1879999999999999,"15min":1.1879999999999999},"max":{"1min":1.6970000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.5129999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.333,"15min":1.343},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.819,"5min":2.077,"15min":2.077},"last":1.403},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.048}]},{"osd":6,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4139999999999999,"15min":1.4299999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.5900000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.462},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.454,"15min":1.46},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.7350000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.3160000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.45,"15min":1.4570000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.911,"5min":2.165,"15min":2.165},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.353,"15min":1.349},"min":{"1min":0.92800000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.796,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.2689999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.29,"15min":1.2669999999999999},"min":{"1min":1.038,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.746,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.4990000000000001,"15min":1.5349999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.839,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.4350000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4790000000000001,"15min":1.5},"min":{"1min":0.98499999999999999,"5min":0.876,"15min":0.876},"max":{"1min":1.7450000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4710000000000001,"15min":1.4670000000000001},"min":{"1min":0.999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8400000000000001,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":1.613}]},{"osd":11,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.615,"15min":1.623},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.889,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.627,"15min":1.623},"min":{"1min":1.1619999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.9890000000000001,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.7030000000000001}]}]},{"osd":6,"up_from":17,"seq":73014444066,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:46:18 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.98399999999999999,"15min":0.96999999999999997},"min":{"1min":0.71399999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.96999999999999997,"15min":0.96799999999999997},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.78100000000000003}]},{"osd":2,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.012,"15min":0.99199999999999999},"min":{"1min":0.71899999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.157},{"interface":"front","average":{"1min":1.2,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.218}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98899999999999999},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.147,"5min":1.1399999999999999,"15min":1.139},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5389999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.91000000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.93999999999999995,"15min":0.92300000000000004},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.244,"15min":1.2330000000000001},"min":{"1min":0.83499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2310000000000001,"15min":1.224},"min":{"1min":0.749,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.194,"15min":1.177},"min":{"1min":0.72499999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.4099999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.343},"min":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.613,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.3},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4159999999999999,"15min":1.4079999999999999},"min":{"1min":0.97099999999999997,"5min":0.877,"15min":0.877},"max":{"1min":1.913,"5min":2.004,"15min":2.004},"last":1.3109999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.5109999999999999,"15min":1.5229999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.879,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.583,"15min":1.5740000000000001},"min":{"1min":1.161,"5min":1.024,"15min":1.024},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.423}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.4790000000000001,"15min":1.458},"min":{"1min":1.1279999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.577,"15min":1.5900000000000001},"min":{"1min":1.002,"5min":0.871,"15min":0.871},"max":{"1min":1.994,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2490000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.665,"15min":1.6559999999999999},"min":{"1min":1.198,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":2.1339999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.663,"5min":1.5469999999999999,"15min":1.5269999999999999},"min":{"1min":1.1910000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.4550000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:44:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":1.413}]}]},{"osd":5,"up_from":17,"seq":73014444067,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298896,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072944,"statfs":{"total":96636764160,"available":96330694656,"internally_reserved":0,"allocated":1576960,"data_stored":970646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.032,"15min":1.026},"min":{"1min":0.73199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0069999999999999,"15min":0.98499999999999999},"min":{"1min":0.69599999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.88500000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.996,"15min":0.98999999999999999},"min":{"1min":0.72499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.0740000000000001,"15min":1.079},"min":{"1min":0.63200000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.357,"5min":1.603,"15min":1.603},"last":1.2749999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.089,"15min":1.0760000000000001},"min":{"1min":0.78700000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.552,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0569999999999999,"15min":1.056},"min":{"1min":0.70299999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.3560000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.143,"15min":1.1399999999999999},"min":{"1min":0.80400000000000005,"5min":0.623,"15min":0.623},"max":{"1min":1.6040000000000001,"5min":1.726,"15min":1.726},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.2709999999999999,"15min":1.2649999999999999},"min":{"1min":0.84399999999999997,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7509999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.93300000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2390000000000001,"15min":1.2370000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.1779999999999999,"15min":1.171},"min":{"1min":0.58699999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2350000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.855,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.478,"5min":1.4179999999999999,"15min":1.4079999999999999},"min":{"1min":0.65400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0189999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:46:10 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.369,"15min":1.3660000000000001},"min":{"1min":0.77600000000000002,"5min":0.755,"15min":0.755},"max":{"1min":2.016,"5min":2.02,"15min":2.02},"last":1.478},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.5329999999999999,"15min":1.5189999999999999},"min":{"1min":1.099,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.3260000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":1.6619999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:46:19 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.468,"15min":1.4590000000000001},"min":{"1min":1.026,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.6439999999999999,"15min":1.6100000000000001},"min":{"1min":1.3029999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.776}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.4750000000000001,"15min":1.4650000000000001},"min":{"1min":0.98299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.6899999999999999,"5min":1.633,"15min":1.623},"min":{"1min":1.101,"5min":1.044,"15min":1.044},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.7370000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.6719999999999999,"15min":1.6639999999999999},"min":{"1min":1.1859999999999999,"5min":0.999,"15min":0.999},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.8,"5min":1.6499999999999999,"15min":1.625},"min":{"1min":1.2749999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.841}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]}]},{"osd":4,"up_from":17,"seq":73014444065,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6870000000000001,"5min":1.786,"15min":1.786},"last":1.929},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.99299999999999999,"15min":1.006},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.2170000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.74099999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0900000000000001,"15min":1.0920000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5660000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.429,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.0109999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.204,"15min":1.224},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5149999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.302,"15min":1.3320000000000001},"min":{"1min":0.83499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4930000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.5049999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.405,"15min":1.4350000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.718,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":2.0579999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.194},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3480000000000001,"5min":1.569,"15min":1.569},"last":1.246}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.80700000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.7709999999999999,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4910000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8979999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.452}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.5580000000000001,"15min":1.5960000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.748,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":2.1840000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.2989999999999999,"15min":1.3280000000000001},"min":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5960000000000001,"5min":1.962,"15min":1.962},"last":1.0720000000000001}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.6499999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.579,"15min":1.611},"min":{"1min":1.0640000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.8109999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.421},{"interface":"front","average":{"1min":1.556,"5min":1.671,"15min":1.6899999999999999},"min":{"1min":1.1899999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":1.9890000000000001,"5min":3.2090000000000001,"15min":3.2090000000000001},"last":1.99}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.601,"15min":1.6160000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.0430000000000001,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.774,"15min":1.802},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.149,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":2.3780000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.6100000000000001,"15min":1.6359999999999999},"min":{"1min":1.129,"5min":1.109,"15min":1.109},"max":{"1min":1.9099999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.742},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.7210000000000001,"15min":1.7270000000000001},"min":{"1min":1.3360000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"max":{"1min":2.093,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":2.214}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.04},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.96699999999999997,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2490000000000001,"5min":1.45,"15min":1.45},"last":1.034}]},{"osd":2,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94099999999999995,"15min":0.93300000000000005},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.93600000000000005,"15min":0.93799999999999994},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.80900000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.103,"15min":1.0960000000000001},"min":{"1min":0.74099999999999999,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5309999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.613},{"interface":"front","average":{"1min":1.135,"5min":1.2350000000000001,"15min":1.2509999999999999},"min":{"1min":0.84099999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4590000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1990000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2370000000000001,"15min":1.2410000000000001},"min":{"1min":0.95499999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.23,"15min":1.24},"min":{"1min":0.78200000000000003,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.627,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3500000000000001,"15min":1.363},"min":{"1min":0.872,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.919,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.73199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.2769999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:46:07 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.327,"15min":1.3160000000000001},"min":{"1min":0.93999999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":0.92000000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.413}]},{"osd":8,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.484,"15min":1.5},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9890000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.5129999999999999,"15min":1.5169999999999999},"min":{"1min":0.99199999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0299999999999998,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.3260000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.589,"15min":1.607},"min":{"1min":0.96399999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9450000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.484},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.571,"15min":1.589},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8640000000000001,"5min":3.327,"15min":3.327},"last":1.8160000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.546,"15min":1.548},"min":{"1min":1.0700000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1240000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.45},{"interface":"front","average":{"1min":1.528,"5min":1.4950000000000001,"15min":1.4890000000000001},"min":{"1min":1.085,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.0819999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.552}]},{"osd":11,"last update":"Sun Sep 15 01:45:58 2024","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5780000000000001,"15min":1.577},"min":{"1min":1.1539999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.482,"15min":1.476},"min":{"1min":0.97599999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.895}]}]},{"osd":2,"up_from":17,"seq":73014444066,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303052,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068788,"statfs":{"total":96636764160,"available":96326438912,"internally_reserved":0,"allocated":1572864,"data_stored":970627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":1.2929999999999999,"15min":1.373},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":10.679,"15min":10.679},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.387},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.49,"5min":10.317,"15min":10.317},"last":0.76900000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.333,"15min":1.407},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.238,"5min":9.673,"15min":9.673},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.304,"15min":1.373},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1459999999999999,"5min":8.9369999999999994,"15min":8.9369999999999994},"last":0.95299999999999996}]},{"osd":3,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95599999999999996,"15min":0.96899999999999997},"min":{"1min":0.42999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.48,"5min":3.6819999999999999,"15min":3.6819999999999999},"last":0.871},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.5669999999999999,"15min":1.657},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4059999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.995}]},{"osd":4,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.575,"15min":1.643},"min":{"1min":0.64400000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6859999999999999,"5min":12.458,"15min":12.458},"last":1.048},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.8500000000000001,"15min":1.944},"min":{"1min":0.86599999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.768,"5min":13.509,"15min":13.509},"last":1.079}]},{"osd":5,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.603,"15min":1.6759999999999999},"min":{"1min":0.59599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.613,"5min":12.188000000000001,"15min":12.188000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.762,"15min":1.841},"min":{"1min":0.83199999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.9399999999999999,"5min":12.295,"15min":12.295},"last":1.331}]},{"osd":6,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.8129999999999999,"15min":1.885},"min":{"1min":0.88300000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.452,"5min":12.864000000000001,"15min":12.864000000000001},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.7989999999999999,"15min":1.8720000000000001},"min":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8080000000000001,"5min":12.9,"15min":12.9},"last":1.2}]},{"osd":7,"last update":"Sun Sep 15 01:45:31 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.147},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.5269999999999999}]},{"osd":8,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.944,"15min":2.0129999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.974,"5min":9.8170000000000002,"15min":9.8170000000000002},"last":1.306},{"interface":"front","average":{"1min":1.595,"5min":1.887,"15min":1.9359999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.859,"5min":9.3140000000000001,"15min":9.3140000000000001},"last":1.5980000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.4430000000000001,"15min":1.425},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.363},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.9379999999999999,"15min":2.0049999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0459999999999998,"5min":9.5549999999999997,"15min":9.5549999999999997},"last":1.4970000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:55 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.921,"15min":1.9750000000000001},"min":{"1min":0.95799999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.8050000000000002,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":1.407},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.8480000000000001,"15min":1.8879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.6890000000000001,"5min":8.8780000000000001,"15min":8.8780000000000001},"last":1.645}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":1,"up_from":17,"seq":73014444065,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1576960,"data_stored":970646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0189999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.01,"5min":0.79000000000000004,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.2010000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.03,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.29,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.91400000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.92300000000000004,"15min":0.90400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.492,"5min":1.595,"15min":1.595},"last":1.147},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0609999999999999,"15min":1.0469999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6020000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.4370000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.117,"15min":1.121},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.583,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1639999999999999,"15min":1.161},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.649,"5min":1.778,"15min":1.778},"last":1.2929999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.64500000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6819999999999999,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1919999999999999,"15min":1.181},"min":{"1min":0.85999999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.696,"5min":1.702,"15min":1.702},"last":1.4079999999999999}]},{"osd":6,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2889999999999999,"15min":1.28},"min":{"1min":1.0569999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.752,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.302,"15min":1.302},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6899999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.248}]},{"osd":7,"last update":"Sun Sep 15 01:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.579}]},{"osd":8,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.7070000000000001,"15min":1.7190000000000001},"min":{"1min":0.76300000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.044,"5min":6.8689999999999998,"15min":6.8689999999999998},"last":1.726},{"interface":"front","average":{"1min":1.635,"5min":1.492,"15min":1.468},"min":{"1min":0.97099999999999997,"5min":0.747,"15min":0.747},"max":{"1min":2.1150000000000002,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.6000000000000001}]},{"osd":9,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.452,"15min":1.421},"min":{"1min":1.02,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.629},{"interface":"front","average":{"1min":1.482,"5min":1.3959999999999999,"15min":1.3819999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.3620000000000001}]},{"osd":10,"last update":"Sun Sep 15 01:45:54 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.486,"15min":1.462},"min":{"1min":1.004,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.548},{"interface":"front","average":{"1min":1.641,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":1.089,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.675}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]}]},{"osd":0,"up_from":17,"seq":73014444065,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1568768,"data_stored":969679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.883,"5min":3.883,"15min":3.883},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.84099999999999997},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":2.0219999999999998,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.5},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97999999999999998,"15min":0.98099999999999998},"min":{"1min":0.624,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.0660000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93500000000000005,"15min":0.93000000000000005},"min":{"1min":0.64500000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.0800000000000001,"15min":1.0640000000000001},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.2929999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.131,"15min":1.141},"min":{"1min":0.75900000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.333,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1160000000000001,"15min":1.0960000000000001},"min":{"1min":0.64700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.248}]},{"osd":6,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.268,"15min":1.262},"min":{"1min":0.91000000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.331},{"interface":"front","average":{"1min":1.097,"5min":1.1060000000000001,"15min":1.1080000000000001},"min":{"1min":0.71599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.597,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.99199999999999999}]},{"osd":7,"last update":"Sun Sep 15 01:46:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.272,"15min":1.2549999999999999},"min":{"1min":0.83899999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.466},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.0820000000000001,"15min":1.0600000000000001},"min":{"1min":0.90800000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.1240000000000001}]},{"osd":8,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4570000000000001,"15min":1.46},"min":{"1min":0.93200000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.129,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5229999999999999,"15min":1.5189999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.6539999999999999}]},{"osd":9,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4750000000000001,"15min":1.4670000000000001},"min":{"1min":1.008,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1760000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.538,"15min":1.536},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.7509999999999999}]},{"osd":10,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.456,"15min":1.4430000000000001},"min":{"1min":0.97099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.526,"15min":1.518},"min":{"1min":0.996,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.8480000000000001}]},{"osd":11,"last update":"Sun Sep 15 01:45:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.337,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8220000000000001,"5min":1.873,"15min":1.873},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.528,"15min":1.5209999999999999},"min":{"1min":0.89400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.2170000000000001,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.7869999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T01:46:22.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T01:46:22.991 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T01:46:22.991 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T01:46:22.991 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T01:46:22.991 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.085 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T01:46:23.085 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T01:46:23.085 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.150 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T01:46:23.150 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T01:46:23.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.216 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T01:46:23.216 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T01:46:23.217 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.293 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T01:46:23.293 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T01:46:23.293 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.385 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T01:46:23.385 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T01:46:23.385 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.449 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T01:46:23.449 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T01:46:23.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.531 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T01:46:23.531 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T01:46:23.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.592 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T01:46:23.592 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-15T01:46:23.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.658 INFO:tasks.ceph.osd.8:Stopped 2024-09-15T01:46:23.658 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-15T01:46:23.658 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.734 INFO:tasks.ceph.osd.9:Stopped 2024-09-15T01:46:23.734 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-15T01:46:23.734 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.804 INFO:tasks.ceph.osd.10:Stopped 2024-09-15T01:46:23.805 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-15T01:46:23.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.891 INFO:tasks.ceph.osd.11:Stopped 2024-09-15T01:46:23.891 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T01:46:23.891 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T01:46:23.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.922 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T01:46:23.922 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T01:46:23.922 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:46:23.923 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.942 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:46:23.943 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T01:46:23.943 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:23.984 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T01:46:23.985 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T01:46:23.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:46:24.002 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T01:46:24.003 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T01:46:24.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-15T01:46:24.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:46:24.025 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T01:46:24.375 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:46:24.375 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T01:46:24.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:46:24.631 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T01:46:24.788 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:46:24.788 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T01:46:24.948 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:46:24.948 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T01:46:25.157 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:46:25.158 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T01:46:25.306 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:46:25.306 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T01:46:25.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:46:25.497 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T01:46:25.616 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:46:25.616 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-15T01:46:25.932 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:46:25.932 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-15T01:46:26.186 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:46:26.186 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-15T01:46:26.359 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:46:26.360 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-15T01:46:26.546 INFO:tasks.ceph:Archiving mon data... 2024-09-15T01:46:26.547 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/data/mon.a.tgz 2024-09-15T01:46:26.547 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-15T01:46:26.554 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.pGPIlk7SC6 2024-09-15T01:46:26.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pGPIlk7SC6 2024-09-15T01:46:26.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.pGPIlk7SC6 2024-09-15T01:46:26.667 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.pGPIlk7SC6 is 302KB 2024-09-15T01:46:26.733 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.pGPIlk7SC6 2024-09-15T01:46:26.740 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/data/mon.b.tgz 2024-09-15T01:46:26.740 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T01:46:26.746 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.pfBHoz6kX5 2024-09-15T01:46:26.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pfBHoz6kX5 2024-09-15T01:46:26.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.pfBHoz6kX5 2024-09-15T01:46:26.879 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.pfBHoz6kX5 is 404KB 2024-09-15T01:46:26.937 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.pfBHoz6kX5 2024-09-15T01:46:26.944 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/data/mon.c.tgz 2024-09-15T01:46:26.944 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-09-15T01:46:26.950 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.r3YbRqeSmR 2024-09-15T01:46:26.951 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.r3YbRqeSmR 2024-09-15T01:46:27.061 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.r3YbRqeSmR 2024-09-15T01:46:27.085 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.r3YbRqeSmR is 404KB 2024-09-15T01:46:27.146 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.r3YbRqeSmR 2024-09-15T01:46:27.153 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T01:46:27.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:46:27.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:46:27.158 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:46:27.173 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-15T01:46:27.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:46:27.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:46:27.200 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:46:27.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:46:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:46:27.243 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:46:27.251 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T01:46:27.252 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi033/crash 2024-09-15T01:46:27.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:46:27.269 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi047/crash 2024-09-15T01:46:27.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:46:27.286 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi155/crash 2024-09-15T01:46:27.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:46:27.304 INFO:tasks.ceph:Compressing logs... 2024-09-15T01:46:27.304 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:46:27.307 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:46:27.309 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:46:27.323 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32364.log 2024-09-15T01:46:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32025.log 2024-09-15T01:46:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32364.log.gz 2024-09-15T01:46:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31701.log 2024-09-15T01:46:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28730.log 2024-09-15T01:46:27.325 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32025.log.gz 2024-09-15T01:46:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28460.log 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31701.log.gz 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33293.log 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28460.log: /var/log/ceph/ceph-client.admin.28730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28460.log.gz 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25987.log 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28730.log.gz 2024-09-15T01:46:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.33293.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31322.log 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33293.log.gz 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32002.log 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31322.log: 0.0%/var/log/ceph/ceph-client.admin.25987.log: -- replaced with /var/log/ceph/ceph-client.admin.31322.log.gz 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33536.log 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25987.log.gz 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28080.log 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32002.log: /var/log/ceph/ceph-client.admin.33536.log: 27.8% -- replaced with /var/log/ceph/ceph-client.admin.33536.log.gz 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32002.log.gz 2024-09-15T01:46:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32012.log 2024-09-15T01:46:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28080.log.gz 2024-09-15T01:46:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25986.log 2024-09-15T01:46:27.331 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22046.log 2024-09-15T01:46:27.331 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22071.log 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30477.log /var/log/ceph/ceph-client.admin.32012.log: 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32012.log.gz 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32780.log 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25986.log: /var/log/ceph/ceph-client.admin.30477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25986.log.gz 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30477.log.gz 2024-09-15T01:46:27.333 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28025.log 2024-09-15T01:46:27.334 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T01:46:27.334 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28025.log: /var/log/ceph/ceph-client.admin.32780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28025.log.gz 2024-09-15T01:46:27.334 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32780.log.gz 2024-09-15T01:46:27.334 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29514.log 2024-09-15T01:46:27.338 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-15T01:46:27.338 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22121.log 2024-09-15T01:46:27.338 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22046.log.gz 2024-09-15T01:46:27.338 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22071.log.gz 2024-09-15T01:46:27.338 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22296.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22146.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22296.log: /var/log/ceph/ceph-client.admin.22121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22296.log.gz 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22121.log.gz 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22146.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.22146.log.gz--verbose 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:-- /var/log/ceph/ceph-osd.9.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22271.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.9.log: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22321.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.11.log: /var/log/ceph/ceph-client.admin.22271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22196.log 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22271.log.gz 2024-09-15T01:46:27.339 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:46:27.340 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22321.log: /var/log/ceph/ceph-client.admin.22196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22171.log 2024-09-15T01:46:27.340 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22321.log.gz 2024-09-15T01:46:27.340 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22196.log.gz 2024-09-15T01:46:27.340 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22096.log 2024-09-15T01:46:27.343 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22171.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-15T01:46:27.343 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22171.log.gz 2024-09-15T01:46:27.343 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22096.log.gz 2024-09-15T01:46:27.343 INFO:teuthology.orchestra.run.smithi155.stderr: 83.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:46:27.344 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22246.log 2024-09-15T01:46:27.344 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 /var/log/ceph/ceph-osd.8.log: --verbose -- /var/log/ceph/ceph-client.admin.22221.log 2024-09-15T01:46:27.344 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22246.log.gz 2024-09-15T01:46:27.346 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T01:46:27.346 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:gzip /var/log/ceph/ceph-mon.b.log: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 83.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T01:46:27.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.4.log: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:46:27.359 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.22221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22221.log.gz 2024-09-15T01:46:27.374 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31376.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30506.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29514.log.gz 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31241.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31376.log.gz 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32807.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30506.log.gz 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31241.log: 28.1% -- replaced with /var/log/ceph/ceph-client.admin.31241.log.gz 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30971.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28806.log 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32807.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.32807.log.gz-5 2024-09-15T01:46:27.375 INFO:teuthology.orchestra.run.smithi033.stderr:--verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30971.log.gz 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29110.log 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28806.log.gz 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32053.log 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30780.log 2024-09-15T01:46:27.376 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29110.log: /var/log/ceph/ceph-client.admin.32053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29110.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32053.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25949.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25684.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr: 26.0% -- replaced with /var/log/ceph/ceph-client.admin.30780.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31714.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25949.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.25949.log.gz--verbose -- 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30264.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25684.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25988.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29646.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31714.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30264.log: gzip -5 /var/log/ceph/ceph-client.admin.25988.log: --verbose -- /var/log/ceph/ceph-client.admin.32435.log 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25988.log.gz 2024-09-15T01:46:27.377 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30264.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29646.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33563.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26006.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32435.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33563.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33563.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.26291.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27598.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26006.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32834.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26291.log.gz 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32753.log 2024-09-15T01:46:27.378 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27598.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29215.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32834.log: /var/log/ceph/ceph-client.admin.32753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32834.log.gzgzip 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.32672.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32753.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.29215.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32011.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29215.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32016.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32672.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32011.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31702.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31716.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32016.log.gz 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29265.log 2024-09-15T01:46:27.379 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31702.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28591.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31716.log: /var/log/ceph/ceph-client.admin.29265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30831.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31716.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr: 27.2% -- replaced with /var/log/ceph/ceph-client.admin.29265.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25676.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28591.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30858.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30831.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25676.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27948.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30858.log.gz 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32321.log 2024-09-15T01:46:27.380 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27948.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.27948.log.gz-- 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31187.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25683.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32321.log: /var/log/ceph/ceph-client.admin.31187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32321.log.gzgzip 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.25997.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31187.log.gz 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28204.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25683.log: gzip /var/log/ceph/ceph-client.admin.25997.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27546.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25997.log.gz -- replaced with /var/log/ceph/ceph-client.admin.25683.log.gz 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28204.log.gz 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32021.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.27546.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31430.log 2024-09-15T01:46:27.381 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27546.log.gz 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32021.log.gz 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30187.log 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32969.log 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31430.log.gz 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30187.log.gzgzip 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.27663.log 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32061.log 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32969.log: /var/log/ceph/ceph-client.admin.27663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31457.log 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32969.log.gz 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27663.log.gz 2024-09-15T01:46:27.382 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33482.log 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32061.log.gz 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32645.log 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31457.log: /var/log/ceph/ceph-client.admin.33482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31457.log.gz 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25682.log 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33482.log.gz 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29001.log 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32645.log.gz 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27541.log 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25682.log.gz 2024-09-15T01:46:27.383 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33347.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29001.log.gz 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27098.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27541.log.gz 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31715.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33347.log.gz 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27989.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27098.log.gz 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32428.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31715.log.gz 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26002.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25964.log 2024-09-15T01:46:27.384 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27989.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32428.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29915.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26002.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26751.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25964.log.gzgzip 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.27475.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29915.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28902.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26751.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27413.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27475.log.gz 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28109.log 2024-09-15T01:46:27.385 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28902.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26670.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27413.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31646.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28109.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33320.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26670.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26457.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31646.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29888.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33320.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26457.log.gz 2024-09-15T01:46:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26857.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29888.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26338.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27889.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26857.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27836.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26338.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31484.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27889.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27836.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27836.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29381.log 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31484.log.gz 2024-09-15T01:46:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30345.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26337.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29381.log: /var/log/ceph/ceph-client.admin.30345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29381.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32440.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30345.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33588.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26337.log: /var/log/ceph/ceph-client.admin.32440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26337.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26411.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32440.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31696.log 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33588.log.gz 2024-09-15T01:46:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21999.log 2024-09-15T01:46:27.389 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26411.log.gz 2024-09-15T01:46:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31696.log.gz 2024-09-15T01:46:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.7.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T01:46:27.426 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33374.log 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.21999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21999.log.gz 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33104.log 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28537.log 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33374.log.gz 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33104.log: /var/log/ceph/ceph-client.admin.26344.log 2024-09-15T01:46:27.434 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33104.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29027.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28537.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33185.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26344.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29027.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32362.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33185.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25589.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32362.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25667.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25589.log.gz 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31565.log 2024-09-15T01:46:27.435 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26852.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25667.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30155.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31565.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.26852.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32369.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26852.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30912.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30155.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.32369.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25666.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32369.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32033.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30912.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32390.log 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25666.log.gz 2024-09-15T01:46:27.436 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32033.log.gz 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26850.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32390.log.gz 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29764.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26778.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26850.log.gz 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31295.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29764.log.gz 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30885.log 2024-09-15T01:46:27.437 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26778.log.gz 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26003.log 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31295.log.gz 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22085.log 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27062.log 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30885.log.gz 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26003.log.gz 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.22085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31052.log 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22085.log.gz 2024-09-15T01:46:27.438 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32326.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27062.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31052.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31052.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32028.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32326.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33401.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26858.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32028.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32429.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33401.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30054.log 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26858.log.gz 2024-09-15T01:46:27.439 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30238.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32429.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32861.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29484.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30054.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30238.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31268.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32861.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.29484.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25679.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29484.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31268.log.gz 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26334.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28323.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29083.log 2024-09-15T01:46:27.440 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26334.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26334.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25679.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29621.log 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28323.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33077.log 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29083.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32915.log 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29621.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28345.log 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33077.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31133.log 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32915.log.gz 2024-09-15T01:46:27.441 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32996.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28345.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27216.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31133.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25675.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32996.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28294.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27216.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30939.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25675.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30022.log 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28294.log.gz 2024-09-15T01:46:27.442 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25687.log 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30939.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26724.log 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30022.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30739.log 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25687.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29940.log /var/log/ceph/ceph-client.admin.26724.log: 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26724.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29967.log 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30739.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32888.log 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29940.log.gz 2024-09-15T01:46:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32332.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29967.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.29967.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.32699.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32888.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26333.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32332.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31403.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32699.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30998.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26333.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27189.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31403.log.gz 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33509.log 2024-09-15T01:46:27.444 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30998.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31079.log 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.27189.log: 76.0% -- replaced with /var/log/ceph/ceph-client.0.27189.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.33509.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27106.log 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33509.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25685.log 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31079.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27106.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32726.log 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25685.log.gz 2024-09-15T01:46:27.445 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25616.log 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30271.log 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32726.log.gz 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27004.log 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25616.log.gz 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17221.log 2024-09-15T01:46:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30271.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32942.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27004.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26616.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.tmp-client.admin.17221.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17221.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33158.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32942.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26370.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26616.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28564.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33158.log.gz 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30590.log 2024-09-15T01:46:27.447 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26370.log.gz 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26018.log 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28564.log.gz 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.30590.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31992.log 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30590.log.gz 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30680.log 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26018.log.gz 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26697.log 2024-09-15T01:46:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31992.log.gz 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30679.log 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30680.log.gz 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33131.log 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26697.log.gz 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30679.log.gz 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29698.log 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33131.log.gz 2024-09-15T01:46:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31025.log 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33239.log 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29698.log: /var/log/ceph/ceph-client.admin.31025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33023.log 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29698.log.gz 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31025.log.gz 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28811.log 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33239.log.gz 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33023.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T01:46:27.450 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33023.log.gz 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26810.log 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28811.log: 83.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28811.log.gz 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31160.log 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31106.log 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26810.log.gz 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29838.log 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31160.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28245.log 2024-09-15T01:46:27.451 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31106.log.gz 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28672.log 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29838.log.gz 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25668.log 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28245.log.gz 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31699.log 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28672.log.gz 2024-09-15T01:46:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32032.log 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25668.log.gz 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31712.log 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31699.log.gz 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25484.log 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32032.log.gz 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29995.log 2024-09-15T01:46:27.453 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31712.log.gz 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.25484.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31538.log 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25484.log.gz 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31700.log 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29995.log.gz 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27296.log 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31538.log.gz 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25686.log 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31700.log.gz 2024-09-15T01:46:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25514.log 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27296.log.gz 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33266.log 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25686.log.gz 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31511.log 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25514.log.gz 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.33266.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26031.log 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33266.log.gz 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.31511.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30429.log 2024-09-15T01:46:27.455 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31511.log.gz 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29433.log 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26031.log.gz 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27747.log 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30429.log.gz 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27594.log 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29433.log.gz 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28780.log 2024-09-15T01:46:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27747.log.gz 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27594.log.gz 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25562.log 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.28780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28780.log.gz 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31214.log 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.25562.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25976.log 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25562.log.gz 2024-09-15T01:46:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22221.log 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31214.log.gz 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31717.log 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25976.log.gz 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33050.log 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.22221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22221.log.gz 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.31717.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26300.log 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31717.log.gz 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33050.log.gz 2024-09-15T01:46:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33428.log 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25458.log 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26300.log.gz 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26295.log 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33428.log.gz 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31705.log 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.25458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25458.log.gz 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30589.log 2024-09-15T01:46:27.459 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26295.log.gz 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27301.log 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31705.log.gz 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28508.log 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30589.log.gz 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31619.log 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27301.log.gz 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:gzip /var/log/ceph/ceph-client.admin.28508.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27352.log 2024-09-15T01:46:27.460 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28508.log.gz 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31592.log 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31619.log.gz 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30581.log 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27352.log.gz 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31592.log.gz 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31349.log 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.30581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31708.log 2024-09-15T01:46:27.461 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30581.log.gz 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26643.log 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31349.log.gz 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.31708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31708.log.gz 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26643.log.gz 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26347.log 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32365.log 2024-09-15T01:46:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29292.log 2024-09-15T01:46:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.26347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26347.log.gz 2024-09-15T01:46:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32365.log.gz 2024-09-15T01:46:27.464 INFO:teuthology.orchestra.run.smithi047.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T01:46:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.29292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29292.log.gz 2024-09-15T01:46:27.548 INFO:teuthology.orchestra.run.smithi033.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T01:46:27.592 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-15T01:46:27.797 INFO:teuthology.orchestra.run.smithi047.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T01:46:27.877 INFO:teuthology.orchestra.run.smithi155.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-15T01:46:27.904 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-15T01:46:27.951 INFO:teuthology.orchestra.run.smithi047.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T01:46:27.957 INFO:teuthology.orchestra.run.smithi033.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T01:46:28.055 INFO:teuthology.orchestra.run.smithi047.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T01:46:28.073 INFO:teuthology.orchestra.run.smithi047.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T01:46:28.075 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-15T01:46:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m0.762s 2024-09-15T01:46:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m2.548s 2024-09-15T01:46:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.111s 2024-09-15T01:46:28.121 INFO:teuthology.orchestra.run.smithi155.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-15T01:46:28.121 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-09-15T01:46:28.121 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m0.807s 2024-09-15T01:46:28.121 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m2.135s 2024-09-15T01:46:28.121 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.149s 2024-09-15T01:46:28.134 INFO:teuthology.orchestra.run.smithi033.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T01:46:28.234 INFO:teuthology.orchestra.run.smithi033.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T01:46:28.334 INFO:teuthology.orchestra.run.smithi033.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T01:46:28.336 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-15T01:46:28.336 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m1.026s 2024-09-15T01:46:28.336 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m3.354s 2024-09-15T01:46:28.336 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.168s 2024-09-15T01:46:28.336 INFO:tasks.ceph:Archiving logs... 2024-09-15T01:46:28.337 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi033/log 2024-09-15T01:46:28.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:46:29.047 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi047/log 2024-09-15T01:46:29.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:46:29.476 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi155/log 2024-09-15T01:46:29.477 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:46:29.919 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T01:46:30.149 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T01:46:30.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:46:30.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:46:30.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:46:30.238 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T01:46:30.247 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T01:46:30.247 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:46:30.250 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:46:30.252 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout:+hv01.front.sepi 67.205.162.81 3 u 9 64 77 0.081 1.068 0.647 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout:+hv02.front.sepi 96.245.170.99 3 u 5 64 77 0.076 0.365 0.326 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout:*hv03.front.sepi 74.6.168.72 3 u 10 64 77 0.103 0.245 0.597 2024-09-15T01:46:30.266 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout:+hv01.front.sepi 67.205.162.81 3 u 1 64 77 0.079 0.118 0.370 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout:*hv02.front.sepi 96.245.170.99 3 u 5 64 77 0.078 0.543 0.387 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout:+hv03.front.sepi 74.6.168.72 3 u 6 64 77 0.099 0.647 0.383 2024-09-15T01:46:30.273 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================== 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:+hv01.front.sepi 67.205.162.81 3 u 4 64 77 0.077 0.250 0.146 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:*hv02.front.sepi 96.245.170.99 3 u 6 64 77 0.062 0.250 0.194 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:+hv03.front.sepi 74.6.168.72 3 u 5 64 77 0.096 0.294 0.121 2024-09-15T01:46:30.275 INFO:teuthology.orchestra.run.smithi047.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T01:46:30.275 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T01:46:30.288 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T01:46:30.289 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T01:46:30.344 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T01:46:30.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T01:46:30.467 INFO:teuthology.task.internal:Duration was 683.877237 seconds 2024-09-15T01:46:30.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T01:46:30.496 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T01:46:30.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:46:30.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:46:30.501 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:46:30.552 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T01:46:30.552 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-09-15T01:46:30.552 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:46:30.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T01:46:30.566 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:46:30.578 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-09-15T01:46:30.578 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:46:30.594 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T01:46:30.595 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:46:30.597 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:46:30.622 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:46:30.660 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T01:46:30.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:46:30.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:46:30.679 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:46:30.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T01:46:30.766 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T01:46:30.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:46:30.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:46:30.798 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:46:30.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T01:46:30.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.840 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-09-15T01:46:30.842 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.855 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-15T01:46:30.857 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-09-15T01:46:30.883 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:46:30.913 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:46:30.919 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:46:30.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:46:30.925 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T01:46:30.939 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T01:46:30.940 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi033 2024-09-15T01:46:30.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:46:30.968 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi047 2024-09-15T01:46:30.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:46:30.997 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905049/remote/smithi155 2024-09-15T01:46:30.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:46:31.022 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T01:46:31.022 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:46:31.025 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:46:31.034 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:46:31.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T01:46:31.051 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T01:46:31.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T01:46:31.116 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T01:46:31.116 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:46:31.118 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:46:31.120 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:46:31.124 INFO:teuthology.orchestra.run.smithi033.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:46 /home/ubuntu/cephtest 2024-09-15T01:46:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:46 /home/ubuntu/cephtest 2024-09-15T01:46:31.128 INFO:teuthology.orchestra.run.smithi155.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:46 /home/ubuntu/cephtest 2024-09-15T01:46:31.130 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T01:46:31.154 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T01:46:31.190 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} duration: 683.8772373199463 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-15T01:46:31.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:46:31.276 INFO:teuthology.run:pass